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SECTION  I 


EXECUTIVE  SUMMARY 


1.0  BACKGROUND 

J 

Since  1967,  the  Department  of  Defense  (DOD)  has  required  its  contractors  to 
have  cost  and  schedule  performance  measurement,  systems  that  satisfy  certain 
criteria.  These  cr i ter i a /'fconTqTned  inJilJDI  70d07?T>are  deemed  necessary  to 
monitor  and  control  contract  performance.  The  criteria,  known  as  the 
Cost/Schedule  Control  Systems  Criteria  (C/SCSC),  are  applicable  to  selected 
contracts  (that  are  not  firm  fixed-price)  within  orograms  designated  as  major 
system  acquisitions.  Over  the_;y£2rrs- since—ItSfiT^ controversies  surrounding 
C/SCSC  and  associated  reporting  requirements  have  existed  within  industry  and 
government.  These  controversies  or  issues  can  be  categorized  under  two 
headings:  M)  whether  the  costs  of  the  criteria  and  associated  reporting 
requirements  outweigh  their  benefits  and  [X  concerns  about  the  specifics  of 
how  these  requirements  have  been  implemented r 

i 

The  Program  Systems  Management  Company  (PSMC^  of  Arthur  0.  Little,  Inc.  (ADL) 
assisted  by  its  wholly  owned  subsidiary,  Opinion  Research  Corporation  (ORC), 
is  currently  under  contract  to  the  office  of  the  Assistant  Secretary  of 
Defense  (Comptroller)  to  conduct  a  survey.  ^The  purpose  of  the  survey  is  to: 

Determine  the  degree  of  acceptance  and  use  of  the  C/SCSC  by  defense 
contractors  and  government  program  managers; 

Identify  problems  and  issues,  the  resolution  of  which  could  lead  to 
improvements  in  the  C/SCSC  and  contract  performance  measurement 
reporting  requirements;  and 

Recommend  policy  changes  that  will  lead  to  these  improvements  and 
could  be  implemented  by  the  Assistant  Secretary  of  Defense 
(Comptroller) . 

The  survey  is  to  be  accomil ished  in  two  phases  --  the  first  phase  being  a 
mailed  questionnaire  and  the  second  phase  being  in-depth  interviews  with  DOD 
and  industry  respondents.  The  first  phase  was  conducted  in  June  1983.  This 
report  contains  the  findings  of  Phase  I,  the  highlights  of  which  are  contained 
in  this  Executive  Summary. 


2.0  SURVEY  POPULATION 


Four  populations  were  surveyed:  DOD  Program  Managers,  Business  Managers, 
Contractor  Program  Managers  and  Contractor  Business  Managers.  Separate 
questionnaires  were  made  for  each  group  reflecting  differences  in  the 
informational  needs  and  perspectives  of  each  group.  The  questionnaires  were 
designed  so  that  most  questions  could  be  answered  by  the  respondents  without 
reference  to  records  and  documents.  We  asked  for  perceptions  and  opinions. 
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i 


Consequently,  many  of  the  survey  results  should  be  interpreted  for  what  they 
are,  namely,  what  the  users  think  about  C/SCSC  and  related  reports.  Depending 
upon  the  specific  issue  being  examined,  these  perceptions  may  or  may  not  be 
consistent  with  "the  truth".  Nevertheless,  these  perceptions  constitute  a 
"reality"  with  which  DOD  must  contend. 

The  response  rate  to  this  survey  exceeded  all  optimistic  expectations  --  over 
80  percent  of  the  eligible  sample.  As  a  consequence,  the  statistical 
confidence  level  one  can  place  in  the  results  approximates  a  census  rather 
than  a  survey. 


3.0  SURVEY  FINDINGS  HIGHLIGHTS 

The  most  important  finding  of  the  survey  was  that  the  majority  of  respondents 
within  each  of  the  four  populations  believe  that  C/SCSC  benefits  to  themselves 
outweigh  its  costs.  However,  the  majorities  were  not  large.  They  ranged  from 
53%  to  62%  in  the  four  populations. 

All  in  all,  business  managers  have  more  favorable  perceptions  towards  C/SCSC 
and  its  associated  reports  than  do  the  program  managers  for  whom  they  work. 
Also,  for  the  most  part,  government  managers  appear  to  have  more  favorable 
perceptions  than  their  contractor  counterparts. 

As  to  the  usefulness  of  the  reports  generated  by  C/SCSC-accepted  systems, 
close  to  four-out-of-five  respondents  rated  the  reports  as  being  either  good 
or  excellent  in  helping  to  determine  the  cost  status  of  their  contracts. 

However,  for  helping  to  determine  aspects  of  contract  status  other  than  cost, 
less  than  half  of  the  government  program  managers  rated  the  reports  as  either 
good  or  excellent.  Government  business  managers,  on  the  other  hand, 
considered  these  reports  to  be  somewhat  more  helpful  to  them.  More  than  half 
of  the  government  business  managers  gave  favorable  ratings  to  the  reports  for 
estimates-at-completion,  cost  impacts  of  known  problems,  problem  traceability, 
and  schedule  status.  More  than  half  of  the  contractor  program  managers  rated 
the  reports  favorably  with  respect  to  estimates-at-completion  and  cost  impacts 
of  known  problems.  As  to  the  timeliness  of  contractor  reports,  most 
government  program  managers  are  not  satisfied,  while  most  government  business 
managers  are  satisfied. 

Despite  the  low  ratings  of  certain  aspects  of  the  reports  and  their  lack  of 
satisfaction  with  the  timeliness  of  the  reports,  when  government  program 
managers  were  asked  whether,  if  they  had  the  choice,  they  would  require  their 
contractors  to  use  C/SCSC-accepted  systems  "as  is"  or  "with  minor 
modifications",  four-out-of-five  government  program  managers  said  they  would. 

A  near  equal  proportion  of  contractor  program  managers  said  they  would  use 
their  C/SCSC-accepted  systems  "as  is"  or  "with  minor  modifications".  However, 
in  terms  of  using  or  requiring  their  C/SCSC-accepted  systems  "as  is"  (that  is. 
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without  any  modifications  at  all),  only  one -out -of -five  contractor  program 
managers  said  they  would  do  so,  and  only  two-out-of-five  government  program 
managers  said  they  would  require  their  systems  to  be  used  "as  is". 

Because  the  various  users  of  C/SCSC  have  somewhat  different  needs  and 
perceptions,  some  of  the  controversies  surrounding  C/SCSC  may  not  be  as 
amenable  to  resolution  as  others.  For  example,  the  difference  in  needs 
between  contractor  and  government  managers  was  readily  apparent  from  two 
findings  solely  related  to  the  government  evaluation  process  for  contractor 
systems.  More  contractor  business  managers  than  not,  feel  that  the  benefits 
to  their  company  of  the  review  process  are  outweighed  by  the  cost  of  the 
review  process.  On  the  other  hand,  government  business  managers  are  nearly 
unanimous  in  their  belief  that  the  review  process  is  necessary  to  the  proper 
functioning  of  the  contractor's  cost  and  schedule  control  system. 

While  firm  conclusions  concerning  these  and  other  findings  of  the  survey  must 
await  in-depth  interview  results  in  Phase  II,  it  appears  that  the  C/SCSC 
approach  to  monitoring  and  controlling  contract  cost  and  schedule  performance 
is  perceived  to  be  useful  by  most  program  and  business  managers  in  both  DOD 
and  industry.  The  results  of  this  survey  make  clear  that  there  is  room  for 
improvement  in  the  application  of  C/SCSC,  as  observed  by  the  participants  and 
users.  In  addition,  although  there  may  be  differences  in  needs  between 
government  and  contractor  managers,  the  results  also  appear  to  indicate  that 
C/SCSC  represents  a  valid  concept  and  approach  and  the  information  derived 
from  C/SCSC-accepted  systems  is  accepted  and  used. 

The  body  of  this  report  contains  detailed  information  on  attitudes  and 
perceptions  concerning  C/SCSC  by  both  DOD  and  industry  officials.  Many  of 
these  findings  are  suggestive  of  directions  for  improvement  in  C/SCSC  policies 
and  practices  which  will  be  explored  in  Phase  II  of  this  contract.  By  its 
very  nature,  a  survey  of  this  type  only  quantifies  --  not  resolves  — 
differences  of  opinion  concerning  matters  ranging  from  the  basic  utility  of 
C/SCSC  in  controlling  acquisition  cost  and  schedule  performance  to  the 
specific  details  of  implementation. 


SECTION  II 


SURVEY  INTRODUCTION 


1.0  C/SCSC  AND  SURVEY  BACKGROUND 

DODI  7000.2  and  the  Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  were 
introduced  in  1967  to  ensure  that  cost/schedule  planning  and  control  systems 
employed  by  contractors  were  fully  integrated,  measured  contract  performance 
objectively,  and  provided  reliable  and  meaningful  data  that  were  derived  from 
a  single  data  base.  Under  the  C/SCSC  approach,  contractors  have  been 
encouraged  to  develop  planning  and  control  systems  suited  to  their  own  needs, 
but  which  satisfy  DOD's  requirements  and  need  for  management  information  and 
visibility. 

An  objective  of  the  C/SCSC  is  to  provide  an  adequate  basis  for  responsible 
decision-making  by  both  contractor  management  and  DOD  components.  To  achieve 
this  objective,  contractors'  internal  management  control  systems  must  provide 
data  which  (1)  indicate  work  progress,  (2)  properly  relate  cost,  schedule  and 
technical  accomplishment,  (3)  are  valid,  timely  and  auditable,  and  (4)  supply 
DOD  managers  with  information  at  a  practicable  level  of  summarization. 

Contractors'  systems  which  comply  with  the  criteria  of  DODI  7000.2  normally 
provide  these  summarized  data  to  the  government  in  the  five  formats  of  the 
Cost  Performance  Reports  (CPR)  prescribed  in  DODI  7000.10.  The  CPR  provides 
(1)  contract  cost/schedule  status  information  for  use  in  making  and  validating 
management  decisions,  (2)  early  indicators  of  contract  cost/schedule  problems, 
and  (3)  effects  of  management  actions  taken  to  resolve  problems  affecting 
cost/schedule  performance. 

Since  1967  DODI  7000.2  has  been  a  requirement  on  major  acquisition  programs. 
More  than  200  contractor  systems  have  been  reviewed  and  found  acceptable.  The 
DOD  and  the  services  have  attempted  to  enhance  and  strengthen  C/SCSC 
implementation  during  this  period.  Activities  and  actions  have  included:  a 
continuing  dialogue  with  contractors  through  industry  associations  (e.g., 

NSIA,  EIA,  AIA,  etc.);  joint  government/ industry  working  groups;  establishment 
of  the  Performance  Measurement  Joint  Executive  Group  (PMJEG),  composed  of  the 
C/SCSC  focal  points  for  each  service;  introduction  of  the  C/SCSC  Joint 
Implementation  Guide,  the  Joint  Surveillance  Guide  and  the  Subsequent 
Application  Review  Program;  revisions  to  DODI  7000.2  and  to  the  C/SCSC  Joint 
Implementation  Guide. 

During  the  period  there  has  also  been  controversy  as  to  the  value  and 
usefulness  of  these  government  requirements.  Examples  of  such  controversies 
include: 


Common  understanding  of  C/SCSC  requirements  by  industry  and 
government ; 

Common  understanding  of  criteria  interpretation  by  government  review 
teams  and  industrial  contractors; 

Degree  of  documentation  requirements,  including  depth  of  reporting; 
and 
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Effectiveness  of  the  use  of  the  information  being  produced  by 
accepted  systems. 


Such  controversies  tend  to  focus  on  the  subject  of  the  management  prerogatives 
of  00D  and  of  industry;  and  elicit  emotion  and  produce  confusion  concerning 
the  acceptance  and  use  of  C/SCSC. 

The  Assistant  Secretary  of  Defense  (Comptroller)  determined  that  a  need 
existed  to  conduct  a  survey  to  identify  the  degree  of  acceptance  and  use  of 
C/SCSC  by  defense  contractors  and  government  program  managers.  Previous 
surveys  and  studies  had  been  made  regarding  C/SCSC  implementation;  however, 
none  had  been  conducted  by  a  professional  and  independent  opinion  research 
firm. 

A  contract  was  awarded  to  Arthur  D.  Little,  Inc.  ( ADL )  in  September  1982  to 
conduct  research  entitled  "Survey  Relating  to  the  Implementation  of 
Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  within  the  Department  of 
Defense  and  Industry."  Management  of  the  survey  is  the  responsibil ity  of 
ADL's  Program  Systems  Management  Company  (PSMC).  PSMC  is  being  assisted  by 
Opinion  Research  Corporation,  a  wholly  owned  subsidiary  of  ADL.  The  major 
purposes  of  the  survey  are  to: 

Determine  the  degree  of  acceptance  and  use  of  the  C/SCSC  by  defense 
contractors  and  government  program  managers; 

Identify  problems  and  issues,  the  resolution  of  which  could  lead  to 
improvements  in  the  C/SCSC  and  contract  performance  measurement 
reporting  requirements;  and 

Recommend  policy  changes  that  will  lead  to  these  improvements  and 
could  be  implemented  by  the  Assistant  Secretary  of  Defense 
(Comptroller) . 


2.0  SURVEY  APPROACH  AND  TASKS 

The  survey  is  being  conducted  in  two  phases.  Phase  I  addressed  the  entire 
population  of  defense  contractor  and  DOD  program  managers  through  the  use  of 
questionnaires.  This  respondent  population  included  all  DOD  programs  which 
had  a  C/SCSC  requirement  and  contractors  with  accepted  systems.  The  results 
of  the  Phase  I  effort  are  the  subject  of  this  report.  Phase  II  will  focus  on 
in-depth  interviews  with  a  selected  sample  of  defense  contractor  and  DOD 
program  managers.  The  following  tasks  were  performed  in  Phase  I: 

Identification  of  the  major  C/SCSC  and  contract  cost  performance 
reporting  related  issues. 

Design,  sample  testing,  and  completion  of  the  questionnaires  based 
on  these  major  issues;  development  of  procedures  for  conducting  the 
survey;  and  identification  of  industry  and  DOD  program  manager 
respondents. 
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Administration  of  the  survey  process.  This  included  mailing  of  the 
questionnaires  to  identified  defense  contractor  and  DOD  program 
managers,  and  follow-up  actions  for  non-respondents. 

Analysis  of  respondent  data  in  the  context  of  the  major  issues 
identified. 

Preparation  of  this  Phase  I  Final  Report.  This  included  developing 
the  Phase  I  conclusions,  and  developing  recommendations  relating  to 
the  in-depth  surveys  to  be  conducted  during  Phase  II. 


3.0  SURVEY  ACTIVITIES 

3.1  Initial  Research  and  Issue  Identification 

The  intial  research  included  interviews  with  government  and  industry 
spokespersons  and  the  review  of  previous  studies  and  surveys.  The  interviews 
included  representatives  of: 

Assistant  Secretary  of  Defense  (Comptroller); 

DOD  Performance  Measurement  Joint  Executive  Group  (PMJEG); 

Defense  Logistics  Agency; 

Defense  Contract  Audit  Agency; 

Defense  Systems  Management  College; 

Sample  of  Army,  Navy  and  Air  Force  government  and  contractor  program 
managers;  and 

Industry  spokespersons  recommended  by  NSIA  Management  Systems 
Subcommittee. 

Identification  and  definition  of  major  C/SCSC  and  contract  cost  reporting 
related  issues  were  derived  from  the  interviews  held  with  industry  and 
government  spokespersons.  The  results  of  these  interviews  produced  a  list  of 
issues  and  concerns  which  were  then  grouped  into  seven  (7)  major  areas  for 
investigation.  These  areas  of  investigation  along  with  their  associated 
issues  or  concerns  became  the  framework  for  survey  design  and  subsequent 
questionnaires. 

The  following  lists  and  briefly  describes  the  seven  (7)  major  areas  of 
investigation. 


This  issue  has  been  the  subject  of  long-standing  controversy.  One  obvious 
difficulty  in  resolving  the  controversy  is  the  determination  of  the 
incremental  cost  between  operation  of  a  C/SCSC-compl iant  system  and  the 
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operation  of  a  system  the  contractor  would  otherwise  use.  Information  about 
the  cost  of  the  latter  alternative  is  not  normally  available.  Therefore,  we 
dealt  with  this  specific  item  as  a  matter  of  respondent  perceptions.  However, 
we  also  included  other  related  items  of  concern  within  this  issue.  The 
specific  items  investigated  were: 

The  benefits  and  effectiveness  of  C/SCSC. 

The  accuracy  of  information  and  usefulness  of  products. 

Cost  versus  benefit. 

Choice  of  whether  to  use  C/SCSC. 

Cost  of  operation. 

(2)  Are  the  criteria  concept,  approach  and  the  criteria  themselves 
appropriate;  and  are  POD  practices  effective? 

The  criteria  approach,  taken  since  1967,  was  a  departure  from  previous  types 
of  requirements.  It  established  a  set  of  criteria  for  management  systems, 
rather  than  specifying  a  specific  system.  One  of  the  purposes,  for  either 
alternative,  would  be  to  ensure  that  the  government  program  office  would  have 
an  adequate  basis  for  keeping  informed  on  the  cost  and  schedule  status  of  its 
contract. 

This  concern  was  examined  in  the  context  of  its  appropriateness,  adequacy  and 
acceptability  as  follows: 

Validity  of  using  a  criteria  concept. 

Appropriateness  of  currently  delineated  criteria. 

Approach  of  ensuring  system  adequacy. 

Effectiveness  of  DOD  practices. 

(3)  Is  the  Government  Review  Process  Effective? 

A  00D  responsibility  in  managing  the  acquisition  of  major  systems  is  to  ensure 
that  visibility  of  contractor's  progress  is  sufficient  to  reliably  indicate 
the  results  being  obtained.  The  C/SCSC  review  process  is  intended  to  help 
assure  this  visibility.  This  issue  addresses  various  elements  of  the 
government  review  process  including  both  Demonstration  Reviews  and  Subsequent 
Application  Reviews  (SARs).  The  following  specific  concerns  were  examined. 

Organizations  participating  on  the  review  teams. 

Need  for  participation  in  the  review  process. 

Duration  of  reviews. 

Size  of  review  teams. 


Number  of  discrepancies  or  corrective  action  items. 

Perceptions  of  review  team  performance. 

Cost  attributable  to  the  review  process. 

Cost  versus  benefit  of  the  review  process. 

Need  for  the  review  process. 

(4)  Are  C/SCSC  consistently  interpreted? 

Contractors  have  indicated  that  interpretation  of  C/SCSC  varies  among  DOD 
review  teams.  This  concern  was  investigated. 

(5)  Are  the  time  allowances  for  certain  key  C/SCSC  actions  realistic? 

This  issue  is  concerned  with  the  elapsed  time  after  contract  award  until  the 
contractor  establishes  the  performance  measurement  baseline  and  until  the 
Demonstration  Review  or  Subsequent  Application  Review  (SAR)  is  held.  The 
issue  addressed  the  following  concerns. 

Time  actually  used  to  establish  performance  measurement  baseline. 

Effect  of  undef initized  contracts  on  establishing  the  baseline. 

Earliest  reasonable  time  to  establish  the  baseline. 

Elapsed  time  after  contract  award  until  conduct  of  C/SCSC  review. 

(6)  Are  Cost  Performance  Reports  and  Internal  Contractor  Cost/Schedule  Data 
Timely  and  Useful"? 

Contractors  have  contended  that  the  government  asks  for  more  reports  (depth 
and  volume)  than  needed  and  that  the  government  doesn't  always  use  what  it 
receives.  The  government  program  managers  often  contend  that  the  Cost 
Performance  Reports  (CPRs)  are  received  too  late  after  the  period  reported  and 
that  they  must  frequently  resort  to  other  more  timely  sources.  The  following 
concerns  were  addressed. 

Depth  of  Cost  Performance  Report  data. 

Volume  of  internal  contractor  reports. 

Timeliness  of  Cost  Performance  Reports 

-  Timeliness  of  internal  contractor  cost/schedule  reports. 

Cost  Performance  Report  use. 

Contractor  use  of  internal  cost/schedule  reports. 
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(7)  Is  government  C/SCSC  surveillance  by  agencies  having  contractor  plant 
cognizance  effective  in  ensuring  that  the  contractor's  system  continues 
to  operate  as  accepted? 

The  government  program  offices  are  dependent  upon  government  plant 
representatives  for  C/SCSC  surveillance.  In  the  interviews  held  at  the  outset 
of  this  survey,  concerns  were  voiced  about  the  following  factors,  which  were 
examined. 

Surveillance  effectiveness. 

Frequency  of  contact  with  the  contractors. 

In-plant  priority  and  expertise. 

3.2  Selection  of  Respondents  and  Questionnaire  Development 

Results  of  the  interviews  indicated  that  survey  respondents  should  be  the 
government  and  contractor  program  managers  and  their  respective  business 
managers.  The  seven  (7)  areas  of  investigation  and  their  associated  issues  or 
concerns,  presented  in  paragraph  3.1,  were  used  for  the  design  of  the  four  (4) 
questionnaires.  Draft  questionnaires  were  developed  and  field  tested  with  two 
government  program  offices  and  three  industrial  contractors.  Revisions  were 
made  as  a  result  of  the  field  testing.  The  final  questionnaires  and  survey 
procedures  were  approved  by  0MB.  Copies  of  the  four  (4)  questionnaires  and 
the  completion  instructions  are  included  in  Appendix  1. 

The  number  of  questions  contained  in  each  of  the  questionnaires  was: 


Government  Program  Manager  22 
Contractor  Program  Manager  27 
Government  Business  Manager  33 
Contractor  Business  Manager  37 


All  questions  were  multiple  choice  except  for  the  final  question  on  each 
questionnaire.  The  final  question  was  optional  and  open-ended.  It  provided 
for  the  respondent's  thoughts  on  any  aspect  of  C/SCSC. 

Program  manager  questionnaires  contained  questions  of  a  general  nature  which 
could  be  completed  quickly  and  would  not  require  reference  to  any  other 
material.  Business  manager  questionnaires  required  responses  at  a  more 
specific  level  of  detail  and  related  to  one  specific  contract.  The  contract 
was  selected  based  on  specific  criteria  contained  in  the  instructions  that 
accompanied  the  questionnaires.  Each  contractor  business  manager  could  select 
only  one  contract  for  the  program  and  each  government  business  manager  could 
select  only  one  contract  for  each  contractor. 

3.3  Respondent  Population 

The  survey  required  that  data  be  collected  from  both  industrial  contractors 
and  government  program  offices.  At  the  time  of  the  survey,  there  were  135  DOD 
programs  that  had  C/SCSC  requirements.  This  entire  universe  was  surveyed. 
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Contract  lists,  with  the  names  and  addresses  of  contractor  and  government 
program  managers  were  provided  by  ODASD  (Cost  &  Audit).  The  total  population 
or  eligible  sample  was: 


TABLE  II-l 
ELIGIBLE  SAMPLE 


Questionnaire 

Military  Department 

Air 

Army  Navy  Force 

No .  No .  No . 

Total 

Eligible 

Sample 

No. 

Government  Program  Manager 

28 

43 

64 

135 

Government  Business  Manager 

48 

64 

78 

190 

Contractor  Program  Manager 

42 

63 

65 

170 

Contractor  Business  Manager 

42 

63 

65 

170 

Totals 

160 

233 

272 

665 

The  numerical  values  in  this  table  show  the  number  of  program  managers 
and  business  managers  in  the  eligible  sample. 


3.4  Survey  Procedures 

Using  the  respondent  lists,  survey  packets  were  mailed  from  Opinion  Research 
Corporation's  (ORC)  mail  center  in  Princeton,  New  Jersey  to  each  contractor 
and  government  program  manager.  Each  packet  contained  a  cover  letter 
explaining  the  purpose  of  the  survey,  the  need  for  the  data,  and  brief 
instructions  (See  Appendix  1).  The  packets  were  mailed  on  June  9,  1983.  The 
requested  return  date  was  June  24,  1983. 

In  order  to  enhance  respondent  cooperation,  the  cover  letter  was  signed  by  the 
Deputy  Secretary  of  Defense.  The  packet  also  contained  the  survey 
questionnaire  in  a  booklet  format  and  a  postage-paid  pre-addressed  envelope  to 
ORC  to  facilitate  its  return.  In  addition,  the  confidentiality  with  which  the 
information  would  be  treated  was  strongly  emphasized. 
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Two  follow-up  procedures  were  used  to  further  enhance  the  response  rate. 
Approximately  five  to  seven  days  after  the  initial  mailing,  reminder  cards 
were  sent  to  all  program  managers.  In  addition,  approximately  three  weeks 
following  the  initial  mailing,  we  contacted  program  managers  by  telephone  who 
had  not  responded  to  remind  them  of  the  survey  and  solicit  their  cooperation. 


4.0  RESPONSE  RATE 

The  response  rate  for  each  of  the  four  (4)  questionnaires  is  shown  by  military 
service  in  the  following  tabulation. 

table  11-2 
RESPONSE  RATE 


Questionnaire 

Military  Department 

Army  Navy  Air  Eorce 

%  %  % 

0VLRALL 

% 

Government  Program  Manager 

100 

74 

80 

83 

Government  Business  Manager 

75 

69 

87 

79 

Contractor  Program  Manager 

93 

81 

86 

86 

Contractor  Business  Manager 

83 

83 

91 

86 

OVERALL 

87 

77 

86 

83 

The  numerical  values  in  this  table  show  the  percent,  for  each  category  in  the 
eligible  sample,  that  responded  to  their  questionnaire. 


The  number  of  respondent  questionnaires  used  for  the  data  tabulations 
discussed  in  Section  III  was  534,  or  80%  of  the  eligible  sample  (665 
questionnaires).  The  overall  response  rate  shown  in  Table  1 1- 2  includes  18 
additional  questionnaires.  Of  these  18  questionnaires,  nine  (9)  responses 
were  comments  only  (i.e.,  responses  solely  to  the  open-ended  question)  ana 
nine  (9)  were  received  after  tabulations  of  the  respondent  questionnaires  were 
completed. 
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The  high  overall  response  rate  provides  a  virtual  census  of  the  population. 
In  part  at  least,  this  high  response  rate  may  be  strong  evidence  of  interest 
and  concern.  An  advantage  of  having  this  high  response  rate  is  that  the 
results  closely  represent  the  views  of  the  total  population,  rather  than 
representing  inferences  regarding  their  views. 


5.0  ANALYSIS  STRUCTURE 

The  Issue/Respondent/Question  Matrix  presented  in  Figure  1  was  used  as  a  guide 
for  the  analysis  presented  in  Section  III  of  this  report.  This  matrix  shows 
the  issues  and  sub-issues  on  the  vertical  axis,  the  four  (4)  questionnaires  on 

the  horizontal  axis,  and  the  individual  question  numbers  within  each  cell  of 
the  matrix. 

Complete  tabulations  of  the  responses  to  each  of  the  questions  in  each  of  the 
questionnaires  are  included  in  Appendix  2.  Cross-tabulation  of  responses 
between  questions  and  statistical  tests  were  accomplished  on  a  selected  basis. 


QUESTIONNAIRES 


Program  Managers 


Business  Managers 


Government 


Contractor 


Contractor 


1.0  C/SCSC  Cost/Benefit 


Benefits  &  Effectiveness 
Accuracy  &  Usefulness 
Cost  vs.  Benefit 
Choice  of  Use 
Cost  of  Operation 


•0  Criteria  Concept 


Validity  of  Concept 
Criteria  Appropriateness 
Ensure  System  Adequacy 
Dot)  Practices 


3.0  C/SCSC  Review  Process 


Participation 
Participation  !Jeed 
Duration 
Team  Size 

System  Deficiencies 
Team  Performance 
Cost 

Cost  vs.  Benefit 
Need  for  Re  vic./s 


4-0  Consistency  of  Interpretation 


5.0  Implement.  Time  Allowances 


Actual  Time  Used  for  Baseline 
Earliest  Reasonable  Time 
Elapsed  Time  to  Review 


6 . 0  Performance  Reporting 

Depth  of  Data 
Internal  Reports 
Timeliness  of  CPRs 
Timeliness  Internal  Reports 
CPR  Use 

Internal  Report  Use 


7.0  Surveillance 


E  f  fecti  \«ness 
Frequency 

Expertise  s  Priority 


Questions  relating  to  population 
descriptors 


1  .2. 3, 4,5, 6,  1,2, 3,4,5, n, 

7,8,9,13  7,8,9,15 


SECTION  III 


ANALYS IS  QF  SURVEY  RESULTS 


introduction 


Thi«  cert  ion  ore sents  the  detailed  analysis  of  the  responses  to  the 
?u«tfonn^«  u  is  organized  hy  the  seven  (7)  ™aJOr  areas  of  revest, 9at, on 
outlined  in  Section  II,  paragraph  3.1. 

For  cross-referencing  to  source  data,  this  analysis  contains  abbreviated 
references  to  the  appropriate  <lu«t*°""aire  „^,,c r^fatiins 

S3M  SS  5=2  ,na9er 

questionnaire,  question  number  21. 
contained  in  Section  II. 

MS?..*  Slr^^So^^P^^t^^eUiS^rSIhaf  , 

%xx  rptre^M  5.1ST5J 

ofPthe  respondent  population.  Some  of  the  data  in  the  appendix  were  also 
useful  in  cPross-tabulations  with  responses  to  questions  in  thi  se ction. 

These  cross-tabulations,  or  relevant  discussion,  are  included  in  the  deta 
analysis  related  to  the  area  of  investigation,  as  appropriate. 

Complete  comparative  tabulations  are  included  in  Appendix  2. 
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1.0  COST  AND  BENEFIT 


1 .1  Background 

The  cost  versus  benefit  of  C/SCSC  has  been  a  long  standing  concern. 
Unfortunately,  objective  and  quantifiable  data  to  assess  costs  and  benefits 
are  not  available.  While  we  may  approximate  the  cost  of  operating  a 
cost/schedule  control  system  that  is  compliant  with  C/SCSC  (as  we  have 
attempted  to  do),  we  do  not  know  the  cost  of  operating  the  same  system  (all 
other  things  being  equal)  without  a  C/SCSC  requirement.  As  in  many  other 
areas,  we  assessed  the  perceptions  of  program  managers  and  business  managers, 
both  contractor  and  government.  Within  their  organizations,  these  people  are 
"closest"  to  the  actual  application  of  C/SCSC  requirements.  They  should 
therefore  have  a  reasonably  clear  understanding  of  program  resources/costs 
associated  with  C/SCSC  applications.  These  managers  are  also  intended  to  be 
the  prime  benefactors  of  C/SCSC-related  disciplines  and  information.  As  a 
result,  we  assume  that  their  perceptions  will  be  a  reasonable  next-best 
substitute  for  the  hard  data  that  are  not  available. 

1 .2  Overview  of  Related  Questions 

Under  the  overall  concern  of  cost  versus  benefit,  we  have  included  the 
questions  which  directly  address  perceptions  or  evaluations  in  four  (4)  areas: 

Perceptions  bearing  solely  on  the  benefits  and  effectiveness  of 
C/SCSC. 

Perceptions  related  to  the  validity  of  information  produced  from 
C/SCSC-accepted  systems,  including  both  Cost  Performance  Reports  for 
the  government-customer  and  internal  data  for  the  contractor. 

Perceptions  of  whether  benefits  outweigh  costs.  This  includes 
whether  C/SCSC  would  be  used,  if  managers  had  a  choice. 

Assessment  of  the  manpower  cost  in  operating  C/SCSC-accepted 
systems;  and  cross-tabluation  of  this  cost  with  how  the  cost  versus 
benefit  question  was  answered. 

1.3  Results 

1.3.1  Benefits  and  Effectiveness  of  C/SCSC 

The  benefits  or  effectiveness  of  C/SCSC  can  be  viewed  as  a  subject  in  itself, 
without  attempting  to  relate  C/SCSC  to  "costs".  This  will  be  discussea  first. 

Government  and  contractor  program  managers  were  asked  whether  they  agreed 
(agree  strongly,  agree  somewhat,  disagree  somewhat,  disagree  strongly,  or 
neither  agree  nor  disagree)  that  a  major  benefit  of  C/SCSC  is  that  it  forces  a 
contractor  to  do  planning  that  otherwise  would  not  have  been  accomplished  as 
thoroughly  ( G PM - 1 3  and  C PM -17).  Contractor  program  managers  were  also  asked 
how  effective  (very  effective,  somewhat  effective,  not  too  effective,  or  not 
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at  all  effective)  their  C/SCSC-accepted  system  was  in  assisting  them  to  keep 
their  program  on  schedule  and  within  cost  (CPM-22).  Government  business 
managers  were  asked  if  they  agreed  (agree  strongly,  agree  somewhat,  disagree 
somewhat,  disagree  strongly,  or  neither  agree  nor  disagree)  that  their 
contractor's  cost  and  schedule  control  systems  were  effective  in  helping 
management  control  contract  performance  (GBM-26).  The  results  are  displayed 
in  Table  1.1. 


Table  i.i 

BENEFITS  AND  EFFECTIVENESS  OF  C/SCSC 


Statement 

Government 

Contractor 

Proq.  Mqr. 

% 

Bus.  Mqr. 

% 

Proq.  Mgr. 
% 

A  Major  Benefit  of  C/SCSC 

77 

★ 

74 

is  More  Thorough  Planning 

Contractor's  System  is 

★ 

80 

71 

Effective 

The  numerical  values  in  this  table  show:  the  percent  of  program  managers 
who  agreed  that  a  major  benfit  of  C/SCSC  is  more  thorough  planning;  the 
percent  of  contractor  program  managers  who  felt  their  system  was 
effective;  and  the  percent  of  government  business  managers  who  agreed 
that  their  contractor's  system  was  effective. 

*  Not  asked. 


Most  government  and  contractor  program  managers  perceived  that  a  major  benefit 
of  C/SCSC  is  more  thorough  contractor  planning  than  otherwise  would  be 
accomplished  (77%  for  government  program  managers  and  74%  for  contractor 
program  managers). 

Most  contractor  program  managers  (71%)  also  evaluated  their  C/SCSC-accepted 
systems  as  being  effective  in  assisting  them  and  their  staffs  for  keeping  the 
program  on  schedule  and  within  cost  limits.  Most  government  business  managers 
also  agreed  (80%)  that  contractor's  cost  and  schedule  control  systems  are 
effective  in  helping  management  to  control  contract  performance. 

It  appears  clear  from  these  data  that  a  major  perceived  benefit  of  C/SCSC  is 
more  thorough  planning.  It  also  appears  clear  that  C/SCSC-accepted  systems 
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are  considered  to  be  effective  by  a  large  majority  of  the  respondents  (71%  for 
contractor  program  managers  and  80%  for  government  business  managers). 

1.3.2  Accuracy  of  Information  and  Usefulness  of  Products. 

Since  the  Cost  Performance  Report  (CPR)  and  internal  contractor  reports  are 
the  useable  products  of  C/SCSC-accepted  systems,  perceptions  regarding  the 
usefulness  of  these  products  and  the  accuracy  or  validity  of  the  information 
they  contain  should  also  be  an  indicator  of  C/SCSC  effectiveness. 

With  this  in  mind,  contractor  program  and  business  managers  were  asked  how 
accurate  (very  accurate,  somewhat  accurate,  not  too  accurate,  or  not  at  all 
accurate)  the  CPR  was  in  portraying  contract  cost  and  schedule  status  (CPM-23 
and  CBM-22).  Government  business  managers  were  asked  whether  they  agreed 
(agree  strongly,  agree  somewhat,  disagree  somewhat,  disagree  strongly,  or 
neither  agree  nor  disagree)  that  the  contractor's  internal  data  gives  the 
program  office  valid  information  at  a  practicable  level  of  summarization 
(G8M-28).  The  results  are  shown  in  Table  1.2. 


TABLE  1.2 

ACCURACY  OF  COST /SCHEDULE  INFORMATION 


Statement 

Government 

Contractor 

Bus.  Mqr. 

% 

Proq.  Mqr. 

% 

Bus.  Mqr. 
% 

CPR  is  accurate 

★ 

91 

92 

Contractor's  internal 
data  is  valid 

80 

★ 

★ 

The  numerical  values  in  the  table  show  the  percent  of  contractor 
respondents  who  felt  that  the  CPR  is  accurate;  and  the  percent  of 
government  business  managers  who  agreed  that  the  contractor's  internal 
data  is  valid. 

*  Not  asked. 


Most  of  the  contractor  program  and  business  managers  (over  90%)  responded  that 
their  CPRs  are  accurate.  Forty  percent  (40%)  of  the  program  managers  and  45% 
of  the  business  managers  responded  that  their  CPRs  were  very  accurate. 
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Most  government  business  managers  (80%)  also  agreed  that  the  contractor's 
internal  data  for  their  contract  gives  the  program  office  valid  information  at 
a  practicable  level  of  summarization. 


It  appears  clear  from  these  responses  that  the  information  produced  from 
C/SCSC-accepted  systems  is  considered  to  be  fairly  accurate  or  valid  by  their 
users. 

To  assess  the  usefulness  of  the  products,  government  program  and  business 
managers  were  asked  to  rate  (excellent,  good,  fair,  poor,  or  no  opinion)  the 
CPRs  they  receive  in  terms  of  how  well  they  help  in  determining  several  items 
of  contract-related  status.  Contractor  program  managers  were  similarly  asked 
to  rate  their  internal  reports.  The  items  are  listed  in  Table  1.3  in  terms  of 
the  percentage  of  managers  who  responded  favorably  (i.e.,  rated  the  item  as 
either  excellent  or  good).  They  are  listed  generally  in  the  order  of  the 
determinations  that  are  believed  to  be  helped  the  most  (GFM-7,  GBM-25  and 
CPM-7) . 


TABLE  1.3 

USEFULNESS  OF  REPORTS 


Contract-Related 

Status 

Government 

Contractor 

Prog.  Mgr. 
(CPR) 

Bus.  Mgr. 
(CPR) 

Prog.  Mgr. 
(Int'l  Reports) 

% 

% 

% 

Cost  Status 

74 

83 

79 

Estimates-at-Completion 

49 

66 

55 

Cost  Impacts  of  Known 

41 

60 

51 

Problems 

Problem  Traceability 

31 

5b 

50 

Schedule  Status 

38 

51 

47 

Problem  Areas  Not 

21 

36 

28 

Previously  Recognized 


The  numerical  values  in  this  table  indicate  the  percent  of  respondents 
that  rated  the  reports  (CPR  or  internal)  as  either  "excellent"  or  "good" 
in  terms  of  how  well  they  helped  determine  the  contract-related  status 
shown. 


How  much  these  products  help  the  managers  depends  on  what  they  are  to  be  used 
for  and  which  manager  the  user  is.  Clearly,  for  the  items  rated,  the  reports 
are  felt  to  be  most  helpful  in  determining  cost  status  and  estimates-at- 
completion.  Fifty  percent  (50%)  or  more  of  both  the  government  business 
managers  and  contractor  program  managers  also  rated  the  reports  as  being 
"excellent"  or  "good"  in  determining  cost  impacts  of  known  problems  and  for 
providing  the  capability  of  tracing  problems  to  their  sources.  Un  the  other 
hand,  these  reports  are  rated  as  being  less  helpful  in  determining  schedule 
status  or  in  identifying  problem  areas  not  previously  known. 

The  data  appear  to  suggest  that  these  reports  are  more  helpful  to  government 
business  managers  and  contractor  program  managers  than  to  government  program 
managers.  The  differences  between  the  government  program  and  business 
managers  should  not  be  unexpected.  The  CPR  is  more  directly  related  to  the 
specific  responsibilities  of  the  government  business  manager.  The  business 
manager  normally  performs  the  CPR-related  work  on  behalf  of  the  program 
manager.  To  explain  the  difference  between  the  government  and  contractor 
program  managers,  it  should  be  kept  in  mind  that  the  question  asked  of  the 
contractor  program  manager  was  related  to  internal  reports  while  the  question 
for  the  government  program  manager  was  related  to  the  CPR. 

1 . 3. 3  Cost  Versus  Benefit 

The  contractor  managers  were  asked  whether  they  agreed  (agree  strongly,  agree 
somewhat,  disagree  somewhat,  disagree  stronqly,  or  neither  agree  nor  disagree) 
that  the  costs  to  their  company  (and  to  DOD),  in  terms  of  time  and  manpower  to 
operate  the  C/SCSC-accepted  system,  are  outweighed  by  the  benefits  to  them  in 
managing  the  contract  (CPM-20  and  CBM-31). 

The  government  managers  were  asked  whether  they  agreed  (agree  strongly,  agree 
somewhat,  disagree  somewhat,  disagree  strongly,  or  neither  agree  nor  disagree) 
that  the  costs  to  their  program  office,  in  terms  of  time  and  manpower  for  all 
C/SCSC-related  activities,  are  outweighed  by  the  benefits  to  them  in  managing 
the  contract  (liPM-16  and  UBM-31). 

The  results  are  presented  in  Table  1.4. 


TABLE  1.4 


COST  VERSUS  BENEFIT 


beneiM  ts 
Outweigh 

Contractor 

Government 

Costs 

Proq.  Mqr. 

% 

Bus.  Mgr. 

% 

Prog.  Mqr. 

% 

Bus.  Mqr. 
% 

Agree 

53 

55 

59 

62 

Disagree 

43 

43 

36 

35 

Neither  Agree 
nor  Disagree 

4 

2 

5 

3 

The  numerical  values  in  this  table  indicate  the  percent  of  respondents 
who  agreed  or  disagreed  with  the  statement  that  the  benefits  of  C/SCSC 
outweigh  its  costs.  For  contractors,  the  responses  related  to  their 
company.  For  the  government,  the  responses  related  to  their  program 
office. 


While  all  four  categories  of  managers  agreed  that  benefits  to  themselves 
outweighed  their  costs,  this  agreement  appears  to  be  slightly  higher  within 
government  managers  than  contractor  managers;  and  slightly  higher  within 
business  managers  than  program  managers. 

1 . 3. 4  Choice  of  Whether  to  Use  C/SCSC 


An  indicator  of  cost  versus  benefit  evaluation  is  whether  program  managers 
would  use  C/SCSC,  when  given  a  choice. 


1.3.4. 1  Current  Contract  Requirements 


Contractor  program  managers  were  asked  whether  they  would  use  their 
C/SCSC-accepted  system  (if  they  had  the  choice);  and  government  program 
managers  were  asked  whether  they  would  require  their  contractors  to  use  their 
C/SCSC-accepted  systems  (if  they  had  the  choice)  (CPM-5  and  GPM-5).  The 
options  and  responses  were  essentially: 


Continue  to  use  (require)  the  C/SCSC-accepted  system  as  currently 
operating  —  contractor  21*  (government  39%)  “* 

Use  (require)  the  current  system  with  minor  modifications  — 
contractor  57%  (government  41%) 


Use  (require)  the  current  system  with  major  modifications;  or  not 
use  (require)  the  system  at  a_]_l_  --  contractor  22%  (government  20%) 


These  results  are  consolidated  in  Table  1.5. 


table  i.s 

CHOICE  OF  WHETHER  TO  USE  C/SCSC 


Use  (Contractor)  or 
Require  (Government) 
the  C/SCSC-Accepted 

Sys  tern 

Contractor 

Prog. 

Mgr. 

(Use) 

Government 

Prog. 

Mgr. 

(Require) 

% 

% 

As  Is 

21 

39 

Minor  Modifications 

57 

41 

Major  Modifications 
or  Not  At  All 

22 

20 

TOTAL 

100 

100 

For  contractor  managers,  the  numerical  values  in  the  table 
indicate  the  percent  of  respondents  that  would  use  their 
C/SCSC-accepted  systems,  if  given  the  choice.  For  government 
managers,  it  is  the  percent  of  respondents  that  would  require 
their  contractors  to  use  a  C/SCSC-accepted  system,  if  given 
the  choice. 


It  appears  clear  that  more  government  program  managers  (39%)  than  contractor 
program  managers  (21%)  prefer  that  C/SCSC-accepted  systems  be  used  as  is.  It 
is  also  clear  that  the  great  majority  of  both  government  program  managers 
(80%)  and  contractor  program  managers  (78%)  do  not  perceive  that  major 
modifications  to  accepted  systems  are  needed.  However,  a  plurality  of 
government  program  managers  (41%)  and  a  majority  of  contractor  program 
managers  (57%)  perceive  the  need  for  at  least  some  minor  modifications. 
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1.3. 4. 2  Other  POD  Contracts  and  Commercial  Work 


Also  related  to  contractor  "choice”  and  its  impact  on  the  issue  of  cost  versus 
benefit,  contractor  program  managers  were  asked  how  often  their  companies  used 
their  C/SCSC-accepted  systems  on  DOD  contracts  when  C/SCSC  was  not  a 
contractual  requirement  (CPM-9).  They  were  additionally  asked  how  often  their 
companies  used  their  C/SCSC-accepted  system  on  commercial  work  (CPM-12). 

For  use  on  other  DOD  contracts,  122  program  managers  indicated  they  could 
answer  (19  of  the  141  respondents  marked  "Don't  know").  Seventy  (70) 
respondents  also  indicated  that  their  companies  had  commercial  work.  Of  these 
70  respondents,  44  program  managers  indicated  they  could  answer  regarding 
commercial  work  (26  of  the  70  respondents  marked  "Don't  know").  The  results 
are  presented  in  Table  1.6. 


TABLE  1 .6 

USE  OF  C/SCSC  WHEN  NOT  REQUIRED 


9 

i 

t 


Use  of  C/SCSC- 
Accepted  System 

For  Other  DOD 
Contracts 

For  Commercial 
Work 

% 

% 

All  of  the  Time 

22 

11 

Some  of  the  Time 

53 

25 

Never 

25 

64 

TOTALS  100 


This  table  shows  responses  of  contractor  program  managers.  It 
indicates  how  often  their  companies  use  their  C/SCSC-accepted 
systems,  when  C/bCSC  is  not  a  contractual  requirement.  Note  that 
the  base  "For  Other  000  Contracts"  is  122  program  managers;  and 
the  base  "For  Commercial  Work"  is  44  program  managers. 


Questions  were  also  asked  regarding  how  often  contractors  used  an  internally 
tailored  earned-value  system  when  earned-value  was  not  a  contractual 
requirement  (CPM-10  andCPM-13).  One-hundred  eighteen  (118)  contractor 
program  managers  answered  regarding  DGu  contracts,  and  41  contractor  program 
managers  answered  regarding  commercial  work.  These  data  are  shown  in  Table 
1.7. 
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TABLE  1.7 


USE  CF  INTERNALLY  TAILORED  EARNEO-VALUE  SYSTEMS 


Use  of  Internally 
Tailored  Earned- 

r or  Other  DOD 
Contracts 

For  Commercial 
Work 

Value  System 

% 

% 

All  of  the  Time 

28 

22 

Some  of  the  Time 

58 

44 

Never 

14 

34 

tutals 

100 

100 

This  table  shows  responses  of  contractor  program  managers.  It 
indicates  how  often  their  companies  used  internally  tailored 
earned-value  systems,  when  such  systems  were  not  a  contractual 
requirement.  Note  that  the  base  for  Other  DOD  Contracts"  is  118 
program  managers;  and  the  base  for  Commercial  Work"  is  41  program 
managers. 


For  contractors  with  C/SCSC-accepted  systems  and  the  capability  for  internally 
tailored  earned-value  systems:  The  data  in  Tables  1.6  and  1.7  suggest  that 
when  given  their  own  choice  of  whether  to  use  either  capability  or  neither 
capability,  a  greater  percentage  will  use  one  of  these  systems  on  DOD 
contracts  than  on  commercial  work.  For  both  DOD  and  commercial  work,  the  data 
also  suggest  that  a  greater  percentage  will  use  an  internally  tailored 
earned-value  system  than  their  C/SCSC-accepted  system. 

We  assume  that  cost  versus  benefit  is  a  key  consideration  in  these  cases.  For 
those  contractors  that  use  their  C/SCSC-accepted  systems  when  not  required, 
they  perceive  the  benefits  as  outweighing  the  costs.  For  those  contractors 
that  do  not  use  their  C/SCSC-accepted  system  when  given  the  choice,  the  costs 
would  outweigh  the  benefits.  Note  that  22%  of  the  program  managers  responoeo 
that  their  company  always  used  its  C/SCSC-accepted  system  on  DOD  work  and  11% 
responded  that  their  company  always  used  it  on  commercial  work.  They  would 
fit  the  first  case  of  perceived  benefits  outweighing  the  costs.  However,  25% 
of  the  program  managers  responded  that  their  company  never  used  its  C/SCSC- 
accepted  system  on  DOD  work  and  64%  responded  that  their  company  never  used  it 
on  commercial  work.  They  would  fit  the  second  case  where  costs  would  outweigh 
benefits. 

1.3.5  Cost  of  Operation 

Contractor  business  managers  were  asked  to  approximate  the  total  number  of 
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employee  man-years  per  year  (i.e.,  cost)  spent  on  operating  their 
C/SCSC-accepted  system  for  their  contract  (CBM-26). 

It  must  be  emphasized  that  this  should  not  be  interpreted  as  a  "delta"  cost 
that  could  be  attributable  solely  to  the  C/SCSC  requirement.  It  is  an 
approximation  of  the  cost  of  operation  for  an  ongoing  system  that  has  been 
accepted  by  the  DOD  as  being  compliant  with  C/SCSC.  If  C/SCSC  would  not  have 
been  a  contract  requirement,  the  contractor  would  still  have  operated  "some" 
cost/schedule  control  system,  with  costs  attributable  to  that  operation. 

This  cost  of  operation  was  also  considered  separate  from  costs  that  might  be 
associated  with  the  C/SCSC  review  process  ( CBM- 12).  Costs  related  solely  to 
the  review  process  are  discussed  in  another  section  of  the  report. 

The  specific  question  (CBM-26)  was: 

"Approximately  how  many  man-years  per  year  of  all  categories  of 
employees  are  spent  on  operating  your  C/SCSC-accepted  system 
for  this  contract  and  analyzing  its  output?" 

The  responses  are  summarized  in  Table  1.8. 


TABLE  1.8 
COST  OF  OPERATION 


Cost  of  Operation  of  Contractor 

C/SCSC-Accepted  System  Bus.  Mgr. 

Man-Years  Per  Year  % 


0  -  5 

18 

6  -  10 

28 

11  -  15 

21 

16  -  20 

9 

21  -  30 

11 

31  or  More 

13 

TOTAL 

100 

The  responses  in  this  table  represent  the  approximations 
provided  by  114  contractor  business  managers.  The  remaining 
22  contractor  business  managers  whose  responses  were 
tabulated  for  this  question  selected  the  response:  "Don't 
know,  too  difficult  to  estimate." 
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For  the  114  contractor  business  managers  who  provided  approximations  of  cost 
(Table  1.8),  eighteen  percent  (18%)  approximated  that  the  cost  of  operation 
was  5  man-years  per  year,  or  less.  Nearly  one-half  (46%)  of  these  respondents 
indicated  a  cost  of  10  man-years  per  year,  or  less.  Approximately  two-thiros 
(67%)  of  these  respondents  indicated  a  cost  of  15  man-years  per  year,  or  less; 
and  approximately  three-quarters  (76%)  of  these  respondents  indicated  a  cost 
of  20  man-years  per  year,  or  less. 

These  responses  (for  cost  of  operation)  were  cross-tabulated  with  the 
contractor  business  manager  responses  which  contained  the  approximate  dollar 
value  of  the  contract  (CBM-5).  The  results  are  shown  in  Table  1.9  for  the 
three  (3)  contract  dollar  value  ranges  of:  Less  Than  $40  Million;  $40-160 
Million;  and  $160  Million  or  more.  The  "break"  points  of  $40  million  and  $160 
million  were  selected  based  solely  on  their  significance  to  C/SCSC  within  the 
Military  Departments.  At  the  time  of  tne  survey,  C/SCSC  was  normally  a 
mandatory  requirement  within  the  departments  for  cost-type  and  fixed-price 
incentive  contracts  which  were  valued  at  $40  million  or  more  for  Research  and 
Development  work  and  $160  million  or  more  for  Production  work. 


Table,  i  .9 

COST  OF  OPERATION  VERSUS  CONTRACT  VALUE 


Cost  of  Operation 
of  C/SCSC-Accepted 
System 

Contract  Dollar  Value 

(Mi  1 1  ions ) 

At  Less 

Least  Than 
“J40 

At  Less 

Least  Than 

Tttr  TTCo 

At  Less 

Least  Than 
$160 

Overal  1 

Man-Years  Per  Year 

% 

% 

% 

% 

0 

-  5 

50 

18 

7 

18 

6 

-  10 

30 

40 

20 

28 

11 

-  15 

20 

26 

18 

21 

16 

-  20 

- 

3 

16 

9 

21 

-  30 

- 

8 

16 

11 

31 

or  More 

- 

5 

23 

13 

TOTAL 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  a  cross-tabulation  of  responses  by 
contractor  business  managers.  The  cross-tabulation  shows  the  approximated 
cost  of  operation  for  C/SCSC-accepted  systems  (Table  1.8)  within  each  of  three 
(3)  contract  dollar  value  ranges. 
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As  should  be  anticipated,  the  data  in  Table  1.9  appears  to  show  that  the  cost 
of  system  operation  is  relatively  elastic  in  relation  to  contract  dollar 
value,  with  lower  costs  associated  with  lower  dollar  value  contracts,  and  vice 
versa.  For  example,  50%  of  the  respondents  with  contract  values  of  less  than 
$40  million  indicated  their  cost  of  operation  as  5  man-years  per  year,  or 
less.  For  contracts  of  at  least  $40  million  but  less  than  $160  million,  the 
percent  of  respondents  who  indicated  the  same  cost  of  operation  (5  man-years 
per  year,  or  less)  dropped  to  18%;  while  for  contracts  of  $160  million  or 
more,  the  percent  of  respondents  was  7%  for  a  cost  of  operation  of  5  man-years 
per  year,  or  less.  Alternatively,  only  20%  of  the  respondents  with  contract 
values  of  less  than  $40  million  indicated  the  cost  of  system  operation  as  11 
man-years  per  year  or  more;  while  the  percent  of  respondents  indicating  11 
man-years  per  year  or  more  increased  to  42%  for  contracts  in  the  $40-$160 
million  range  and  to  73%  for  contracts  of  $160  million  or  more. 

The  responses  to  cost  of  operation  were  also  cross-tabulated  with  the 
contractor  business  manager  responses  for  cost  versus  benefit  (CBM-31).  The 
results  of  this  cross-tabulation  are  shown  in  Table  1.10. 


TABLE  1.10 

COST  OF  OPERATION  VERSUS  COST/BENEFIT 


Cost  of  Operation 

Benefits  Outweigh  Costs 

of  C/SCSC-Accepted 
System 

Agree 

Disagree 

Neither  Agree 

Nor  Disagree 

Total 

Man-Years  Per  Year 

% 

%  ‘ 

i 

% 

0  -  5 

71 

29 

0 

100 

6  -  10 

63 

34 

3 

100 

11  -  15 

50 

50 

0 

100 

16  or  More 

41 

59 

0 

100 

Don't  Know 

59 

36 

5 

100 

Overall 

55 

43 

2 

100 

The  numerical  values 

in  this  table  show 

a  cross- 

tabulation  of  responses  by 

contractor  business  managers.  The  cross-tabulation  shows  the  approximated 
cost  of  operation  for  C/SCSC-accepted  systems  (Table  1.8)  with  whether  the 
respondent  agreed  or  disagreed  that  benefits  of  the  system  outweighed  its 
costs  (Table  1.4). 
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The  pattern  of  the  data  in  Table  1.10  appears  to  suggest  that  perceptions  of 
whether  benefits  outweigh  costs  are  relatively  elastic  and  inversely  related 
to  the  cost  of  system  operation  (i.e.,  the  higher  the  approximated  cost  of 
operation  for  a  contractor's  system,  the  lower  the  agreement  by  that 
contractor  business  manager  that  benefits  outweigh  costs). 

For  example,  when  the  cost  of  system  operation  was  approximated  as  0-5 
man-years  per  year  (the  lowest  cost-range  shown  in  the  table),  71 %  of  the 
contractor  business  managers  agreed  that  benefits  outweighed  costs.  But  when 
the  cost  was  identified  as  16  or  more  man-years  per  year  (the  highest 
cost-range  shown  in  the  table),  only  41%  of  the  business  managers  agreed  that 
benefits  outweighed  costs.  (Overall  agreement  that  benefits  outweighed  costs, 
as  shown  in  this  table  and  previously  shown  for  contractor  business  managers 
in  Table  1 .4,  was  55%. ) 

This  pattern  appears  consistent  with  expectations. 

1.4  Summary 

Based  on  the  respondent  data,  we  conclude  that: 

C/SCSC-accepted  systems  are  considered  to  be  effective  by  a  large 
majority  of  the  respondents  (contractor  program  and  government 
business  managers).  A  large  majority  of  contractor  and  government 
program  managers  perceive  that  a  major  benefit  of  C/SCSC  is  more 
thorough  contractor  planning  than  otherwise  would  be  accomplished. 

CPUs  and  related  internal  contractor  reports  are  considered  to  be 
fairly  accurate  or  valid  by  their  users.  The  reports  are  considered 
to  be  most  helpful  in  determining  contract  cost  status.  They  are 
also  helpful  to  government  business  managers  and  contractor  program 
managers  for  estimates-at-completion,  determining  cost  impacts  of 
known  problems,  and  tracing  problems  to  their  sources,  in  that 
order.  They  are  not  as  helpful,  however,  in  these  areas  to 
government  program  managers. 

All  four  categories  of  managers  agreed  that  C/SC5C  benefits  to 
themselves  outweighed  its  associated  costs.  However,  the  strength 
of  this  belief  was  less  than  that  relating  solely  to  C/5CSC 
effectiveness.  When  program  managers  are  given  the  choice  of 
whether  they  would  require  or  use  current  C/SCSC-accepted  systems  as 
is,  or  with  some  modification,  more  government  program  managers  than 
contractor  program  managers  would  require  or  use  these  systems  as 
is.  Nevertheless,  a  plurality  of  government  program  managers  and  a 
majority  of  contractor  program  managers  perceive  the  need  tor  at 
least  some  minor  system  modifications.  The  great  majority  of  both 
government  and  contractor  program  managers  ao  not  perceive  the  neea 
for  major  modifications. 
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Eighteen  percent  (18%)  of  the  contractor  business  managers 
approximated  the  cost  of  operating  their  C/SCSC-accepted  systems 
(not  the  cost  of  C/SCSC)  as  5  man-years  per  year  or  less.  Nearly 
one-half  (46%)  approximated  this  cost  as  10  man-years  per  year  or 
less;  while  two-thirds  (67%)  and  three-fourths  (76%)  approximated 
the  cost  as  15  and  20  man-years  per  year  or  less,  respectively. 
Patterns  in  the  data  suggest  that  perceptions  of  whether  benefits 
outweigh  costs  appear  to  be  inversely  related  to  the  cost  of 
operating  C/SCSC-accepted  systems.  Additionally  the  data  suggests 
that  these  costs  vary  directly  with  the  dollar  value  of  the 
applicable  contract.  These  patterns  in  the  data  appear  consistent 
with  expectations. 
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2.0  CRITERIA  CONCEPT  AND  APPROACH 


2.1  Background 

The  use  of  C/SCSC  and  the  criteria  approach  has  been  DUD  policy  since  1967 
(approximately  16  years).  The  degree  of  acceptance  of  this  policy  and  its 
practice  was  examined  on  a  general  level.  More  specific  concerns  are  covered 
in  later  paragraphs. 

A  related  concern  on  this  general  level  is  whether  the  policy  intention  of 
using  criteria  has  actually  evolved  into  the  practice  of  requiring  a  specific 
"system";  or  whether  D00  practices  may  be  overly  specified  and  procedural. 

2. 2  Overview  of  Related  Questions 

To  determine  perceptions  and  acceptance  of  the  C/SCSC  approach  on  a  general 
basis,  we  asked  questions  which  covered  a  range  of  subject  matter  from  the 
concept  of  "criteria"  to  D00  practices  in  implementing  C/SCSC.  Similar 
questions  were  asked  of  program  managers  and  business  managers,  both 
government  and  contractor.  The  specific  questions  related  to  perceived 
assessments  regarding: 

The  validity  of  using  a  criteria  concept. 

The  appropriateness  of  DuU's  currently  delineated  criteria  as  the 
basis  for  evaluating  contractor  systems. 

Whether  the  approach  of  ensuring  that  contractor  systems  are 
adequate  is  as  good  or  better  than  other  approaches  for  obtaining 
reliable  cost/schedule  data. 

The  effectiveness  of  UOD  practices  in  ensuring  that  contractor 
systems  are  adequate. 

2. 3  Results 

To  address  these  objectives,  respondents  were  presented  with  an  attitudinal 
statement  pertaining  to  each  of  the  four  (4)  topics.  Respondents  were  asked 
to  indicate  the  degree  to  which  they  agreed  or  disagreed  with  each,  using  a 
five-point  scale  (agree  strongly,  agree  somewhat,  disagree  somewhat,  disagree 
strongly  or  neither  agree  nor  disagree).  The  attitudinal  statements  ano 
respondents  are  briefly  identified  for  each  of  the  four  (4)  topics  and  the 
results  are  presented  in  the  tables  that  follow. 

2.3.1  Criteria  Concept,  the  Criteria,  POD  Approach  and  POD  Practices 

First,  regarding  the  validity  of  using  a  criteria  concept,  goverment  program 
managers  and  contractor  program  and  business  managers  were  asked  whether  the 
concept  of  using  criteria  rather  than  specifying  a  single  system  for  all 
contractors  is  valid  lCPM-10,  CPM-14  and  IBM-32). 
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Second,  regarding  the  appropriateness  of  the  currently  delineated  criteria, 
government  program  and  business  managers,  and  contractor  program  and  business 
managers  were  asked  whether  the  current  DOD  criteria  are  appropriate  for 
evaluating  contractor  systems  (OPM-12,  OBM-27,  CPM-16  and  CBM-34). 

Third,  regarding  the  approach  of  ensuring  system  adequacy,  government  program 
managers  and  contractor  program  and  business  managers  were  asked  whether 
ensuring  system  adequacy  is  as  good  or  better  than  other  approaches  for 
obtaining  cost/schedule  status  (bPM-11,  CPM-15  and  CBM-33). 

Fourth,  for  effectiveness  of  DOD  practices,  government  program  and  business 
managers,  and  contractor  program  and  business  managers  were  asked  whether  DOD 
practices  were  effective  in  ensuring  that  contractor  systems  are  adequate  for 
cost/schedule  reporting  (bPM-14,  bBM-29,  CPM-18  and  CBM-30). 

Table  2.1  below  shows  the  overall  results  from  the  four  (4)  topics. 


TABLE  2.1 


CRITERIA  CONCEPT  AND  APPROACH 


Criteria  Concept 
is  Valid 

77 

* 

82 

88 

Current  Criteria  are 
Appropriate 

71 

77 

63 

b9 

Ensuring  System  Adequacy 
is  as  Good  or  Better 
than  Other  Approaches 

73 

★ 

77 

77 

DOD  Practices  are  Effective 

74 

82 

67 

75 

The  numerical  values  in  this  table  indicate  the  percent  of  respondents 
that  agreed  (either  strongly  or  somewhat)  with  the  specific  statement. 


*  Not  asked. 

Note:  For  quality  control  purposes,  the  first,  second  and  fourth 
questions  above  were  asked  in  the  negative  (e.g.:  are  not  valid;  are  not 
appropriate;  and  are  not  effective).  For  presentation  purposes,  the 
responses  have  been  reversed  to  show  the  positive  questions. 


It  appears  evident  from  the  table  that  agreement  among  all  managers  questioned 
is  most  favorable  towards  the  validity  of  the  criteria  concept.  While  most  of 
the  managers  consider  the  current  criteria  to  be  appropriate,  this  topic 
elicits  the  weakest  favorable  response,  especially  among  contractors. 
Approximately  one-in-three  contractor  managers  disagreed  with  this  statement. 
Within  each  category  of  manager  (government  program  manager,  contractor 
program  manager  and  contractor  business  manager),  there  appears  to  be  a 
relatively  steep  drop  in  strength  of  agreement  from  the  manager's  responses 
regaroing  the  validity  of  the  criteria  concept.  This  drop  in  strength  of 
agreement  appears  most  evident  among  the  contractor  managers  (i.e.:  from  82% 
to  63%  for  program  managers;  and  from  88%  to  69%  for  business  managers). 
Apparently,  more  disagreement  exists  regarding  the  currently  delineated 
criteria  than  whether  the  concept  itself  is  valid. 

The  differences  in  the  relative  strength  of  agreement  between  the  validity  of 
the  criteria  concept  and  the  appropriateness  of  the  current  criteria  should 
not  be  unexpected. 

Valid  cost/schedule  information  has  been  and  should  continue  to  be  necessary 
in  order  to  assess  contract  cost/schedule  status.  Some  policy  and  practice  on 
this  matter  should  be  expected  as  a  fact-of-1 ife.  We  also  shoula  expect  a 
criteria  approach  to  appeal  to  both  000  and  its  contractors.  We  should  expect 
that  the  notion  of  UOD  specifying  a  set  of  criteria  which  its  contractors  must 
meet  to  be  a  preferable  alternative  to  requiring  contractors  to  use  some 
specific  system.  A  criteria  "approach"  satisfies  DOU  needs  by  promising  valid 
information  and  it  satisfies  the  000  contractors  by  promising  minimum 
interference  or  minimum  imposition  of  unneeded  internal  management  systems. 

However,  we  also  should  expect  differences  between  000  and  contractors  to 
arise  regarding  a  criteria  approach  as  we  move  away  from  the  conceptual  notion 
and  move  more  towards  practical  implementation.  This  expectation  appears  to 
be  consistent  with  the  data. 

Most  managers  agreed  that  the  approach  of  ensuring  an  adequate  basis  (system 
adequacy)  for  cost/schedule  information  is  as  good  or  better  than  other 
approaches  for  obtaining  cost  and  schedule  status.  Most  managers  similarly 
agreed  that  000  practices  are  effective  in  accomplishing  this  objective, 
although  contractor  program  managers  appear  to  be  less  in  agreement  than  the 
other  three  (3)  categories  of  managers. 

Regarding  whether  000  practices  are  considered  overly  specified  and 
procedural,  it  should  be  noted  that  agreement  appeared  reasonably  strong  and 
consistent  between  all  categories  of  managers  that  000  practices  are 
effective.  Agreement  appeared  even  stronger  in  this  area  than  whether  the 
current  criteria  are  appropriate.  This  might  be  considered  somewhat 
surprising,  since  000  "practices"  regarding  C/SCSC  also  could  be  considered 
relatively  less  conceptual  and  more  towards  practical  implementation  than  a 
statement  or  delineation  of  a  set  of  criteria. 

The  data  also  appear  to  suggest  that  business  managers  have  more  favorable 
views  regarding  these  topics  than  the  program  managers  for  whom  they  work. 

This  is  consistent  with  the  results  related  to  the  usefulness  of  cost/schedule 
reports  (discussed  in  paragraph  1.3.2  above). 
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2.3.2  Differences  between  Demonstration  Reviews  and  Subsequent  Application 
Reviews  (SaRs) 

The  contractor  business  manager  responses  for  each  of  the  four  (4)  topics  were 
cross-tabulated  with  the  type  of  review  (either  demonstration  or  SAR)  that  the 
government  performed  on  the  specific  contract  (CBM-1).  This  was  done  to 
determine  if  there  might  be  differences  in  contractor  perceptions  regarding 
the  criteria  concept  and  approach  related  to  whether  the  contractor  had  just 
undergone  a  demonstration  or  a  SAR.  The  results  are  shown  in  Table  2.2. 


TABLE  2.2 

PERCEPTIONS  OF  CRITERIA  CONCEPT  AND  APPROACH  BY  TYPE  OF  REVIEW 


Contractor  Business  Managers 

Statement  Demonstration  SAR  Overal 1 


%  %  % 


Criteria  Concept 
is  Valid 

82 

92 

88 

Current  Criteria 
are  Appropriate 

55 

75 

69 

Ensuring  System  Adequacy 
is  as  Good  or  Better 
than  Other  Approaches 

76 

78 

77 

DOD  Practices  are 

Effective 

61 

81 

75 

The  numerical  values  in  this  table  indicate  the  percent  of  contractor 
business  managers  that  agreed  (either  strongly  or  somewhat)  with  the 
specific  statement. 


The  data  in  Table  2.2  suggest  more  favorable  perceptions  regarding  the 
criteria  concept,  the  current  criteria  and  the  effectiveness  of  DOD  practices 
for  contractors  who  had  undergone  SARs  than  for  contractors  who  had  just 
undergone  the  demonstration  process.  The  differences  in  the  table  are 
statistical ly  significant  at  the  95%  confidence  level  for  both  the  validity  of 
the  criteria  concept  and  the  effectiveness  of  DOD  practices. 

While  the  review  process  will  be  examined  in  more  detail  in  the  next  section 
of  this  report,  it  should  be  noted  here  that  a  SAR  is  normally  performed  when 
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a  contractor  already  has  a  previously  accepted  system,  it,  therefore,  appears 
from  the  data  that  contractors  perceive  the  criteria  concept,  the  current 
criteria,  and  DOD  practices  more  favorably  after  their  C/SCSC-accepted  system 
has  been  operating  for  some  time,  rather  than  shortly  after  their  initial 
exposure. 

2. 4  Summary 

Based  on  the  respondent  data,  we  conclude  that: 

The  criteria  concept  is  considered  valid  by  a  large  majority  of  the 
respondents  (government  program  managers;  and  contractor  program  and 
business  managers). 

Most  managers  in  all  four  (4)  categories  (government  program  and 
business  managers;  and  contractor  program  and  business  managers) 
considered  the  currently  delineated  criteria  appropriate  as  the 
basis  for  evaluating  contractor  systems.  While  most  contractor 
managers  considered  the  criteria  appropriate,  the  percent  was  not  as 
high  as  for  their  counterpart  government  managers.  In  addition;  for 
contractor  managers,  there  appeared  to  be  less  agreement  that  the 
currently  delineated  criteria  are  appropriate  than  that  the  criteria 
concept  is  valid. 

A  large  majority  of  the  respondents  (government  program  managers; 
and  contractor  program  and  business  managers)  agreed  that  the 
approach  of  ensuring  that  an  adequate  basis  exists  for  keeping  the 
program  office  informed  is  as  good  or  better  than  other  approaches 
to  achieving  the  same  objective. 

A  large  majority  in  all  four  (4)  categories  (government  program  and 
business  managers;  and  contractor  program  and  business  managers) 
considered  00U  practices  effective  in  ensuring  that  an  adequate 
basis  exists  in  a  contractor's  system  for  reporting  cost/schedule 
status.  While  a  large  majority  of  contractor  managers  considered 
DUO  practices  effective,  the  percent  was  not  as  high  as  for  the 
government  managers.  Un  a  general  level,  it  does  not  appear  that 
DOD  practices  are  overly  specified.  Specific  DOD  practices  as  they 
relate  to  the  C/SCSC  review  process  and  depth  of  reporting  are 
discussed  as  separate  concerns  in  later  sections  of  this  report. 

For  the  validity  of  the  criteria  concept  and  the  effectiveness  of 
DOD  practices,  contractor  business  managers  have  significantly  more 
favorable  perceptions  after  undergoing  the  Subsequent  Application 
Keview  process  than  after  undergoing  the  demonstration  process. 
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3.0  C/SCSC  REVIEW  PROCESS 


3. 1  Background 

A  key  practice  in  the  C/SCSC  approach  is  the  review  process.  Since  this 
process  can  be  viewed  as  a  government  evaluation  of  contractor  systems  and 
practices,  it  provides  the  circumstances  for  adversarial  relationships  and 
mutual  concerns. 

As  part  of  the  C/SCSC  concept  and  approach,  contractors  demonstrate  their 
systems  in  operation  to  000  review  teams.  The  government  review  team 
evaluates  whether  to  accept  (or  "validate")  the  contractor's  demonstrated 
system  as  compliant  with  C/SCSC.  This  is  referred  to  as  the  demonstration 
process  (or  sometimes,  the  "validation"  of  the  contractor's  system). 

If  the  contractor's  system  was  previously  accepted  as  C/SCSC-compliant  on  an 
earlier  contract  of  the  same  type  ( i . e . ,  Research  and  Development  work  versus 
Production  work),  then  the  demonstration  and  review  practices  for  the  current 
contract  are  normally  less  extensive  than  for  the  earlier  contract  that  lea  to 
the  previous  acceptance.  This  less  extensive  review  process  is  referred  to  as 
a  Subsequent  Application  Review  (SAR). 

We  examined  contractor  and  DOU  experience  and  perceptions  regarding  the 
conduct  and  effectiveness  of  demonstration  reviews  and  i>ARs . 

3.2  Overview  of  Related  Questions 


As  part  of  this  examination  of  the  C/SCSC  review  process,  we  included 
questions  which  directly  addressed  experience,  assessments  and  perceptions  in 
the  eight  (8)  areas  listed  below.  The  respondents  for  these  questions  were 
the  business  managers,  both  contractor  and  government.  Where  appropriate, 
similar  questions  were  asked  of  both  categories  of  managers. 


Experience  and  perceptions  relating  to  the  DOD  organizational 
composition  of  review  teams. 


Experience  regarding  duration  of  reviews. 

Experience  regarding  review  team  size. 

Experience  regarding  the  need  for  contractor  corrective  action  prior 
to  acceptance. 

Perceptions  of  review  team  performance  and  effectiveness. 

Assessments  of  contractor  cost  solely  attributable  to  the  conduct  of 
the  reviews. 

Perceptions  of  whether  the  benefits  of  the  review  process  outweigh 
its  costs. 

Perceptions  of  the  need  for  the  review  process. 
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3.3  Results 


The  C/SCSC  review  process  normally  entails  several  in-plant  visits  by  DQu 
review  teams.  For  the  following  questions  which  relate  to  organizational 
participation,  review  duration,  team  size,  number  of  discrepancies  and  team 
performance,  the  respondents  were  asked  to  use  as  a  reference  only  the  one 
visit  they  considered  to  have  been  the  key  C/SCSC  review  for  the  particular 
contract. 

3.3.1  Organizations  Participating  on  the  Review  Team 

Government  and  contractor  business  managers  were  asked  to  indicate  which 
government  organizations  participated  in  the  review  (G8M-12B  and  12C  and 
CBM-11B).  The  responses  are  shown  separately  in  Table  3.1  for  Demonstration 
Reviews  and  SARS. 


TABLE  3.1 

PARTICIPATION  IN  THE  REVIEW 


Demonstrations 

SARS 

Participating 

Gov '  t 

Contractor 

Gov't 

Contractor 

Organizations 

Bus.  Mqr 
% 

.  Bus.  Mgr. 

% 

Bus.  Mgr, 
% 

l  Bus 

•Jlsr 

% 

Gov't  Plant  Representative 

95 

88 

89 

96 

(eg.,  OCAS,  AFPRO,  NAVPRO) 

Program  Office 

88 

88 

97 

93 

Service  Focal  Point 

11 

85 

70 

82 

(HQ  DARCUM,  HQ  NAVMAT, 

HQ  AFSC) 

DCAA 

74 

88 

76 

82 

Commodity  Command 

60 

33 

52 

48 

(eg.,  MICUM,  ASD,  NAVA  IK) 

Other 

12 

18 

15 

18 

The  numerical  values  in  this 

table  show 

the  percent  of 

respondents 

that 

indicated  participation  of  the  organizations  listed.  The  respondent  data  for 
government  business  managers  (other  than  "Program  Office")  includes  only  those 
reviews  in  which  the  program  office  indicated  its  own  participation  (GBM-12B). 
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As  apparent  from  the  data,  the  reviews  teams  are  organized  to  include  the 
participation  of  several  organizations. 

3.3.2  Need  for  Participation  in  the  Review  Process 

Government  business  managers  were  asked  to  indicate  how  necessary  (very 
necessary,  somewhat  necessary,  not  necessary)  to  the  C/SCSC  review  process  was 
the  participation  of  the  various  government  organizations  (GBM-14A,  14B,  14C, 
140  and  14£).  The  responses  are  shown  in  Table  3.2. 


TABLE  3.2 

NEED  FOR  PARTICIPATION 


Participating 

Organization 

Necessary 
- % - 

Program  Office 

99 

Contract  Administration  Office 
(e.g.,  OCAS,  AFPRO) 

97 

DCAA 

94 

Service  Focal  Point 

(HQ  DARCOM,  HQ  NAVMAT,  HQ  AFSC) 

74 

Commodity  Command 

(e.g.,  ASD,  NAVAIR,  MICOM 

73 

The  numerical  values  in  this  table  show  the  percent  of  respondents  that 
indicated  participation  of  the  listed  organization  was  necessary  (very 
necessary  or  somewhat  necessary)  to  the  C/SCSC  review  process.  Respondents 
were  government  business  managers,  whether  their  program  office  participated 
in  the  review  or  not. 


The  data  were  separately  obtained  for  Demonstrations  and  SARs.  However,  there 
was  no  apparent  difference  in  the  pattern  of  the  results  between  the  two  types 
of  reviews.  Therefore,  the  data  are  shown  as  aggregated. 

It  is  apparent  from  the  data  that  most  government  business  managers  feel  that 
participation  is  necessary  for  each  of  those  organizations  which  currently 
provide  members  to  review  teams.  From  the  business  manager's  perspective, 
there  was  almost  unanimous  agreement  regarding  the  necessity  for  participation 
of  the  program  office,  contract  administration  office  and  the  DCAA. 
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3.3.3  Duration  of  C/SCSC  Reviews 


Government  and  contractor  business  managers  were  asked  to  indicate  the 
duration  of  the  review  in  workdays  (GBM-12D  and  CBM-11C).  The  responses  are 
shown  separately  in  Table  3.3  for  Demonstration  Reviews  ana  SwRs. 


TABLE  3.3 

DURATION  OF  C/SCSC  REVIEWS 


Demonstrations 

SARs 

Workdays 

Government 

Bus.  Mgr. 

- 

Contractor 
Bus.  Mqr. 

% 

Government 
Bus.  Mqr. 

% 

Contractor 
Bus.  Mgr. 

■  %  - 

0-5 

26 

21 

76 

68 

6-10 

26 

24 

16 

29 

11  -  15 

39 

37 

7 

2 

More  Than  15 

9 

18 

1 

1 

TOTAL 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  that 
selected  each  number  of  workdays,  for  the  duration  of  their  applicable  review. 


As  evident  in  the  data,  most  Demonstrations  are  longer  in  duration  than  SARs. 

For  Demonstrations,  approximately  one-half  of  the  reviews  (52%  according  to 
government  business  managers  ana  45%  according  to  contractor  business 
managers)  are  completed  within  two-weeks  (1U  workdays  or  less).  Wore  than 
four-fifths  of  the  reviews  (91%  according  to  government  business  managers  and 
82%  according  to  contractor  business  managers)  are  completed  within  three 
weeks. 

The  data  suggest  that  contractor  business  managers  perceive  the  Demonstration 
Reviews  as  longer  in  duration  than  do  their  government  counterparts.  A 
possible  explanation  is  that  government  team  members  may  frequently  be 
released  from  team  participation  after  their  team  assignments  are  completed. 
This  could  include  team  members  from  the  business  manager's  office.  However, 
the  team  leadership  may  remain  in  the  contractor's  plant  to  complete  followup 
actions  and  report  writing.  These  activities  may  involve  interface  with  the 
contractor  business  manager. 
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Most  SARs  are  completed  within  one  (1)  week  (0-5  workdays)  while  nearly  all 
SARs  (92%  according  to  government  business  managers  and  97%  according  to 
contractor  business  managers)  are  completed  within  two  (2)  weeks. 

As  shown  in  the  data,  the  duration  of  the  reviews  appears  to  be  somewhat 
variable.  This  result  should  not  be  completely  unexpected.  The  set  of 
considerations  attendent  to  any  given  review  is  probably  unique  from  all  other 
reviews.  For  example,  while  the  "shorter"  SARs  are  normally  conducted  solely 
for  contractor  systems  that  have  been  previously  accepted  for  the  same  type  of 
work  ( i . e . ,  Research  and  Development  versus  Production),  SARs  may  sometimes 
(under  conditions  specified  by  the  military  services)  be  conducted  in  lieu  of 
the  "longer"  Demonstration  Reviews.  Such  "hybrid"  SARs  may  be  expected  to  be 
longer  in  duration  than  normal  SARs.  In  addition.  Demonstrations  and  SARs  are 
sometimes  jointly  conducted  on  more  than  one  contract  or  program.  These 
reviews  may  be  expected  to  be  longer  in  duration  than  those  conducted  for  one 
contract  or  program. 

3.3.4  Size  of  Review  Team 

Government  and  contractor  business  managers  were  asked  to  indicate  the  size  of 
the  government  review  team  (G8M-12E  and  CBM-11D).  The  responses  are  shown 
separately  in  Table  3.4  for  Demonstration  Reviews  and  SARs. 


TABLE  3.4 

SIZE  OF  REVIEW  TEAM 


Number 

of 

Persons 

Demonstrations 

SARs 

Government 
Bus.  Mgr. 
i  — 

Contractor 
Bus.  Mgr. 

% 

Government 
Bus.  Mgr. 
t 

Contractor 
Bus.  Mgr. 

% 

O 

1 

23 

15 

33 

23 

8  -  11 

37 

43 

44 

45 

12  -  15 

28 

15 

18 

18 

16  -  19 

7 

12 

5 

10 

More  Than  19 

5 

15 

0 

4 

TOTAL 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  that 
indicated  each  number  of  persons,  as  the  size  of  the  applicable  review  team. 
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Based  on  the  respondent  data,  the  predominant  review  team  size  is  8-11 
persons.  For  Demonstration  Reviews,  approximately  60%  of  the  respondents 
indicated  the  review  team  size  was  11  persons  or  less.  For  SARS, 
approximately  three-fourths  (77%)  of  the  government  business  managers  and 
two-thirds  (68%)  of  the  contractor  business  managers  indicated  that  the  review 
team  size  was  11  persons  or  less. 

The  pattern  of  the  data  within  each  of  the  two  groups  of  respondents 
(government  business  managers  and  contractor  business  managers)  suggests  that 
the  number  of  persons  on  Demonstration  Review  teams  is  greater  than  the  number 
on  SAR  teams.  Since  Demonstration  Reviews  are  normally  more  extensive  than 
SaRs,  this  should  be  expected. 

3.3.5  Discrepancies  or  Corrective  Action  Items. 

Government  and  contractor  business  managers  were  asked  to  approximate  the 
number  of  discrepancies  or  corrective  action  items  that  were  identified  by  the 
DOD  Review  Team  (GBM-12F  and  CBM-11E).  The  responses  are  shown  in  Table  3.5. 


TABLE  3.5 

DISCREPANCIES  IDENTIFIED  DURING  THE  REVIEW 


Demonstrations 

SARs 

Number 

of 

Discrepancies 

Government 
Bus.  Mqr. 

% 

Contractor 
Bus.  Mqr. 

% 

Government 
Bus.  Mqr. 

% 

Contractor 
Bus.  Mqr. 

% 

0  -  10 

51 

45 

67 

67 

11-20 

24 

14 

21 

22 

21  -  30 

14 

14 

7 

6 

More  Than  30 

11 

27 

5 

5 

total 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  that 
indicated  the  number  of  discrepancies  listed. 
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Based  on  the  respondent  data  for  Demonstration  Reviews,  approximately  one-half 
of  the  business  managers  (51%  for  government  and  45%  for  contractors) 
indicated  the  number  of  discrepancies  as  10  or  less  during  the  C/SCSC  reviews 
of  their  contracts;  while  for  SARs,  approximately  two-thirds  (67%)  of  both 
government  and  contractor  business  managers  indicated  the  number  of 
discrepancies  as  10  or  less.  For  government  business  managers,  75%  indicated 
20  discrepancies  or  less  during  Demonstrations  while  88%  indicated  20 
discrepancies  or  less  during  SARs.  For  contractor  business  managers,  59% 
indicated  20  discrepancies  or  less  during  Demonstrations  while  89%  indicated 
20  discrepancies  or  less  during  SARs. 

The  data  suggest  that  the  need  for  contractor  corrective  actions  prior  to 
government  acceptance  is  less  during  SARs  than  during  Demonstration  Reviews. 
This  conclusion  appears  compatible  with  the  more  favorable  perceptions  that 
contractors  have  during  SARs  than  during  the  demonstration  process  regarding 
the  criteria  concept,  the  current  criteria  and  DUD  practices  (paragraph  2.3.2 
above) . 

In  addition  we  should  expect  a  lesser  number  of  corrective  actions  during 
SARs.  Unlike  contractors  undergoing  Demonstration  Reviews,  contractors 
undergoing  SARs  will  always  have  a  previously  accepted  system  and  have  had 
prior  exposure  to  one  or  more  C/SCSC  review  teams.  Contractors  undergoing 
SARs  should  not  be  as  likely  to  have  significant  system  discrepancies  which 
would  have  been  identified  and  corrected  during  the  earlier  Demonstration 
process.  These  contractors  should  also  be  better  prepared  to  avoid 
discrepancies  based  on  their  past  experience  with  DOD  C/SCSC  review  procedures. 

3.3.6  Perceptions  of  Review  Team  Performance 

Contractor  business  managers  were  asked  to  rate  (excellent,  good,  fair,  poor, 
or  no  opinion)  the  C/SCSC  review  team's  performance  on  six  (6) 
characteristics.  The  characteristics  are  listed  in  Table  3.6.  Percentages 
are  shown  for  combined  ratings  of  "excellent"  and  "good"  (CBM-11F). 
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TABLE  3.6 


REVIEW  TEAM 

PERFORMANCE 

Review  Team 

Demonstration 

SAR 

Characteristic 

Review 

% 

X 

Working  Relationships 

94 

93 

Overall  Professionalism  and 

Effectiveness 

76 

86 

Leadership  and  Control 

76 

82 

Thoroughness 

73 

84 

Common  Sense  and  Flexibility 

55 

68 

Technical  Qualifications 

52 

71 

The  numerical  values  in  this  table  indicate  the  percent  of  contractor  business 
manager  respondents  who  rated  the  team  characteristic  as  either  "excellent"  or 
"good". 


The  data  show  that  the  majority  of  contractor  business  managers  perceive  that 
D00  review  teams  do  a  good-to-excel lent  job.  Ratings  are  highest  for  working 
relationships.  There  are  also  large  majorities  (good-to-excel lent )  for 
overall  effectiveness,  leadership  and  thoroughness.  The  smallest  majorities 
are  in  the  characteristics  of  flexibility  and  technical  qualifications. 

For  each  character istic  (except  for  working  relationships  where  the 
good-to-excel lent  ratings  included  over  90%  of  the  respondents),  the  percent 
of  respondents  who  gave  a  combined  rating  of  excellent  and  good  is  higher  for 
SARs  than  for  Demonstration  reviews. 

Since  the  number  of  required  corrective  action  items  appears  to  be  higher  for 
Demonstrations  than  for  SARs  (paragraph  3.3.5  above),  it  might  appear  that  the 
level  of  the  review  team  rating  could  be  inversely  affected  by  the  number  of 
discrepancies  identified  by  the  review  team.  However,  cross-tabulations  of 
review  team  ratings  with  number  of  discrepancies  did  not  support  this 
inference.  There  were  no  apparent  differences  in  review  team  ratings  related 
to  number  of  discrepancies. 


3.3.7  Cost  Attributable  to  the  Review  Process 


Contractor  business  managers  were  asked  to  estimate  the  number  of  man-days 
spent  by  their  company  in  support  of  the  review  process  over  and  above  the 
normal  expenditures  for  the  day-to-aay  operation  of  their  cost/schedule 
control  system  ^CtsM-12).  The  responses  are  shown  in  Table  3.7. 


TABLE  3.7 

REVIEW  PROCESS  SUPPORT 


Review  Process 
Support 
in  Man-days 

Demonstrations 

SARs 

% 

% 

0  -  25 

6 

20 

26  -  50 

13 

14 

51  -  100 

6 

19 

101  -  200 

41 

27 

201  -  300 

6 

6 

More  Than  300 

28 

14 

TOTAL 

100 

100 

The  responses  in  this  table  represent  estimates  provided 
by  contractor  business  managers. 


The  data  appear  to  show  that  more  support  is  required  for  the  demonstration 
process  than  for  the  SAR  process.  This  conclusion  is  consistent  with  the  data 
that  show  Demonstration  Reviews  to  be  longer  in  duration  than  SARs  (paragraph 
3.3.3  above).  Specifically,  100  man-days  or  less  was  estimated  for  25%  of  the 
demonstration  processes  and  for  53%  of  the  SAR  processes.  Two-hundreo  (2UU) 
man-days  or  less  was  estimated  for  6b%  of  the  demonstration  processes  and  for 
80%  of  the  S«R  processes. 

3.3.8  Cost  Versus  Benefit  of  the  Review  Process 

Contractor  business  managers  were  asked  if  the  benefits  to  their  company  of 
the  review  process  outweighed  its  costs  (CBM-13).  Responses  are  shown  in 
Table  3.8. 
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table  3.8 


COST  VERSUS  BENEFIT  OF  THE  REVIEW  PROCESS 


Benefits 

Outweigh  Costs 

Demonstrations 

% 

SAR" 

% 

Yes 

52 

30 

No 

39 

55 

No  Opinion 

9 

15 

TOTAL 

100 

100 

The  numerical  values  in  this  table  indicate  the  percent  of 
contractor  business  manager  respondents  answering  Yes,  No, 
or  No  opinion  regarding  whether  the  benefits  of  the  review 
process  outweigh  its  costs. 


These  results  appear  to  show  more  favorable  perceptions  towards  Demonstration 
Reviews  than  towards  SaRs.  The  contractor  business  manager  perceptions  of 
whether  the  benefits  outweigh  costs  for  Demonstration  Reviews  appears  to  be 
approximately  the  same  as  their  assessments  regarding  whether  the  overall 
benefits  of  their  C/SCSC-accepted  system  outweigh  its  operating  costs  (see 
Table  1.4). 

The  more  unfavorable  perception  regarding  the  benefits  of  SARS  versus  their 
cost  appears  surprising  if  solely  considering  some  of  the  previously  cited 
results,  such  as  number  of  required  corrective  actions  (Table  3.5),  rating  of 
review  team  performance  (Table  3.6),  cost  of  the  review  process  (Table  3.7), 
and  perceptions  regarding  the  criteria  concept,  the  current  criteria  and  UOU 
practices  (Table  2.2).  The  results  for  each  of  these  would  appear  to  infer 
more  favorable  perceptions  of  SARs  than  Demonstrations. 

However,  we  also  cross-tabulated  the  contractor  business  manager  responses  for 
benefits  versus  costs  with  responses  that  contained  estimates  of  man-days 
spent  in  support  of  the  review  process  (Table  3.7)  ( CBM- 12). 

For  SaRs,  there  was  no  apparent  relationship  regarding  whether  the  responses 
of  benefits  versus  costs  varied  with  differences  between  the  estimated  cost  of 
the  revie.  irocess.  The  relationship  appeared  to  be  relatively  inelastic. 
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For  Demonstrations  the  relationship  appeared  to  be  less  inelastic.  Six  out  of 
eight  of  the  respondents  (75%)  who  estimated  the  cost  of  the  review  process  as 
100  man-days  or  less  indicated  that  the  benefits  outweighed  costs;  while  only 
42%  of  the  respondents  who  estimated  the  cost  of  the  review  process  as  greater 
than  100  man-days  indicated  that  the  benefits  outweighed  costs.  While  some 
relationship  is  suggested,  the  small  size  of  the  sample  indicating  100 
man-days  or  less  precludes  any  definitive  conclusion  that  such  a  relationship 
exists. 

It  should  be  noted  that  the  purpose  of  both  Demonstration  Reviews  and  SARs  is 
for  contractors  to  demonstrate  to  the  government  that  their  systems  comply 
with  C/SCSC  and  that  they  operate  as  advertised.  If  this  government  "show-me" 
requirement  did  not  exist,  contractors  could  conceivably  still  operate 
C/SCSC-compl iant  systems.  The  reviews  therefore  do  not  serve  contractor 
purposes  nearly  as  much  as  they  serve  the  government's  purpose  of  verifying 
that  the  contractor  systems  are  providing  valid  cost/schedule  information. 

Under  these  conditions,  both  the  lack  of  benefit  perceived  by  contractors  for 
SARs  and  the  inelasticity  of  this  perception  to  the  cost  of  SAR  support  is 
understandable.  For  Demonstration  Reviews,  contractors  will  probably  be 
experiencing  some  concepts  and  practices  related  to  C/SCSC  that  are  new  to  the 
company.  They  may  consider  some  of  these  concepts  and  practices  as 
beneficial.  This  benefit  is  not  as  likely  to  occur  during  SARs.  While  both 
Demonstration  Reviews  and  SARs  may  be  viewed  by  contractors  as  unwanted  or 
unneeded  “tests",  contractors  apparently  view  the  Demonstration  Reviews  (and 
the  initial  exposure  to  C/SCSC  that  is  normally  associated  with  these  reviews) 
as  more  beneficial  to  them  than  SARs. 

3. 3.9  Need  for  the  Review  Process 

Government  business  managers  were  asked  how  necessary  (very  necessary, 
somewhat  necessary,  not  necessary)  the  C/SCSC  review  process  is  to  the  proper 
functioning  of  the  contractor's  cost  and  schedule  control  system  (GBM-15). 

The  responses  are  shown  in  Table  3.9. 
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NEEL)  FOR  REVIEW  PROCESS 


Necessity  of 

Review  Process 

Demonstration 

% 

5a£ 

% 

Very  Necessary 

83 

69 

Somewhat  Necessary 

15 

29 

Not  Necessary 

2 

2 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of 
government  business  manager  respondents  that  selected  the 
choices  shown. 


It  is  apparent  from  the  data  that  there  was  nearly  unanimous  agreement  among 
government  business  managers,  regarding  the  need  for  the  review  process. 
Ninety-eight  percent  (98%)  indicated  the  review  process  was  necessary  (very 
necessary  or  somewhat  necessary).  Only  2%  of  the  respondents  felt  that  the 
review  process  was  "not  necessary".  Based  on  the  scale  rating  (very  necessary 
versus  somewhat  necessary),  agreement  regarding  the  need  for  the  demonstration 
process  appeared  to  be  slightly  stronger  than  for  the  SAR  process. 

The  near  unanimous  result  is  significant  from  two  aspects. 

First,  the  respondents  are  from  the  program  offices  that  are  responsible  for 
using  and  interpreting  the  information  reported  from  contractor  systems.  They 
are  not  primarily  concerned  with  C/SCSC-compl iance  for  its  own  sake. 

Second,  the  strength  of  their  viewpoint  highlights  a  difference  in  needs 
between  the  government  and  its  contractors,  as  represented  by  their  business 
manager  respondents.  Government  business  managers  appear  to  be  saying  that 
the  review  process  is  essential;  they  need  to  have  verification  that  the 
contractor's  system  is  operating  properly.  From  earlier  cited  contractor 
business  manager  responses,  regarding  whether  benefits  of  the  review  process 
outweigh  its  cost,  contractor  business  managers  appear  to  be  saying  that  this 
verification  is  not  necessarily  for  their  benefit  (paragraph  3.3.8  above). 
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3.4.  Summary 

Based  on  the  respondent  data,  we  conclude  that: 

000  Review  teams  are  normally  organized  to  include  participation  of 
the  program  office,  government  plant  representatives,  resident  DCAA 
representatives,  service  focal  point  office  and  commodity  command. 

Most  government  business  managers  feel  that  review  team 
participation  is  necessary  for  each  of  the  organizations  which 
currently  provide  team  members.  There  was  nearly  unanimous 
agreement  regarding  the  necessity  for  program  office,  contract 
administration  office  and  DCAA  participation. 

For  Demonstrations,  approximately  one-half  of  the  reviews  are 
completed  within  two  weeks  and  more  than  four-fifths  are  completed 
within  three  weeks.  Most  SARs  are  completed  within  one  week. 

The  predominant  number  of  review  team  members  is  8-11  persons.  For 
Demonstration  Reviews,  approximately  60%  of  the  respondents 
indicated  the  review  team  size  was  11  persons  or  less.  For  SaRs, 
approximately  three-fourths  of  the  government  business  managers  and 
two-thirds  of  the  contractor  business  managers  indicated  the  review 
team  size  was  11  persons  or  less. 

For  Demonstrations,  10  discrepancies  or  less  (requiring  contractor 
corrective  actions)  are  identified  during  approx imately  one-half  of 
the  reviews.  For  SARs,  10  discrepancies  or  less  are  identified 
during  approximately  two-thirds  of  the  reviews. 

The  majority  of  contractor  business  managers  feel  that  D0D  review 
teams  do  a  good-to-excel lent  job  when  evaluating  their  company’s 
C/5CSC  application.  The  largest  majorities  (good-to-excel lent )  were 
for  three  characteristics:  overall  effectiveness,  leadership  and 
thoroughness.  The  smallest  majorities  were  for  two  characteristics: 
flexibility  and  technical  qua  1 1 f  icat  ions  . 

For  costs  solely  attributable  to  the  conduct  of  C/5C5C  reviews,  the 
contractor  business  manager  responses  appear  to  show  that  more 
support  is  required  for  the  demonstration  process  than  for  SARs. 

For  example,  1UU  man-days  or  less  was  estimated  for  25%  of  the 
demonstration  processes  and  for  53%  of  the  SAk  processes. 

-  The  majority  of  contractor  business  managers  indicated  that  they 
believe  that  the  benefits  to  their  company  of  the  demonstration 
process  outweigh  its  costs  (costs  solely  attributable  to  the 
reviews).  However,  for  SARs,  the  majority  of  contractor  business 
managers  felt  that  the  benefits  did  not  outweigh  the  costs. 
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Government  business  managers  were  nearly  unanimous  (98%)  in 
indicating  that  the  C/SCSC  review  process  was  necessary.  This  was 
true  for  both  the  demonstration  process  and  SAR  process,  based  on 
the  scale  ratings,  agreement  regarding  the  need  for  the 
demonstration  process  appeared  to  be  slightly  stronger  than  for  the 
SAR  process. 
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4.0  CONSISTENCY  OF  C/SCSC  INTERPRETATION 

4.1  Background 


Since  each  C/SCSC  review  team  (in  terms  of  the  people  on  the  team)  is 
different  from  all  other  teams,  we  should  expect  contractor  concern  regarding 
the  consistency  of  C/SCSC  interpretations  and  evaluations  between  different 
teams . 

Uniformity  of  criteria  interpretation  (under  similar  circumstances)  should  be 
important  for  efficient  government  implementation  and  equity  of  treatment 
between  contractors.  In  practice,  the  responsibility  for  uniform 
implementation  of  C/SCSC  and  evaluation  of  contractor  systems  is  assigned  by 
DOD  to  the  major  commands  in  the  military  services.  Each  major  command 
headquarters  has  a  focal  point  which  organizes  and  directs  review  teams,  ana 
provides  interpretation  of  how  the  criteria  apply  to  specific  contractor 
systems. 

Based  on  contractor  experience  with  DOU  review  teams,  we  examined  the  degree 
of  contractor  concern  regarding  consistency  of  C/SCSC  interpretations. 

4.2  Related  Question 

The  specific  question  was  addressed  to  contractor  business  managers.  It 
related  to  perceived  assessments  regarding: 

Consistency  of  the  review  team's  C/SCSC  interpretations  with  other 
00 D  review  teams'  interpretations. 

4.3  Results 


Contractor  business  managers  were  asked  (yes,  no,  or  no  opinion)  whether  their 
review  team's  C/SCSC  interpretations  were  reasonably  consistent  with  other  000 
review  teams'  interpretations  (CBM-11G).  The  responses  are  shown  in  Table  4.1. 
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TABLE  4.1 


CONSISTENCY  UF  C/SCSC  INTERPRETATIONS 


C/SCSC  Interpretations 
are  Reasonably  Consistent 

Demonstrations 

% 

SMs 

% 

Yes 

58 

71 

No 

15 

8 

No  Opinion 

27 

21 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of 
contractor  business  manager  respondents  answering  yes,  no 
or  no  opinion  regarding  whether  C/SCSC  interpretations  are 
reasonably  consistent. 


For  the  respondents  who  expressed  their  opinion  (yes  or  no),  79%  of  those  who 
went  through  Demonstration  Reviews  and  90%  of  those  who  went  through  SARs 
indicated  reasonable  consistency  with  other  DOD  review  teams. 

It  should  be  noted  that  contractors  undergoing  SARs  always  would  have  been 
through  an  earlier  demonstration  process.  The  data  therefore  appears  to 
suggest  that  those  respondents  likely  to  have  been  through  prior  reviews  have 
higher  agreement  that  DUD  review  teams  are  reasonably  consistent  in  their 
C/SCSC  interpretations  (than  those  respondents  who  may  not  have  been  through 
prior  reviews). 

4.4  Summary 

Based  on  the  respondent  data,  we  conclude  that: 

Most  contractor  business  managers  feel  that  their  review  team's 
C/SCSC  interpretations  were  reasonably  consistent  with  other  DUD 
review  teams’  interpretations.  The  pattern  of  the  data  appears  to 
suggest  that  there  is  higher  agreement  regarding  team  consistency 
for  those  contractors  undergoing  SARs  than  for  those  undergoing 
Demonstration  Reviews. 
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5.0  time  allowances  for  implementing  c/scsc 


5.1  Background 

The  time  allowances  for  implementing  C/SCSC  on  a  contract  and  accomplishing 
the  C/SCSC  review  process  have  frequently  been  cited  as  concerns. 

A  Defense  Acquisition  Regulation  (DAR)  contract  clause  is  normally  used  in 
contracts  when  C/SCSC  compliance  is  required.  The  clause  indicates  that  the 
contractor  should  be  prepared  to  demonstrate  its  cost/schedule  control  system 
within  ninety  (90)  calendar  days  after  contract  award  (or  a  different  number 
of  days,  as  otherwise  agreed  to  by  the  contracting  parties). 

For  C/SCSC  implementation  and  an  effective  demonstration  and  verification  (via 
a  Demonstration  or  a  Subsequent  Application  Review),  the  performance 
measurement  baseline  must  be  established.  If  the  performance  measurement 
baseline  for  the  contract  is  not  established,  then  many  of  the  key  contractor 
subsystems  cannot  be  adequately  demonstrated  or  verified  for  that  contract. 

5.2  Overview  of  Related  Questions 


To  determine  experience  and  perceptions  regarding  time  allowances,  we  included 
questions  which  directly  addressed  experience  and  assessments  in  the  areas 
listed  below.  The  respondents  for  these  questions  were  the  business  managers, 
both  contractor  and  government.  As  in  prior  sections,  we  asked  similar 
questions  of  both  categories  of  managers. 

Experience  regarding  when  the  performance  measurement  baseline  was 
established,  including  contracts  that  were  not  definitized  at 
contract  award. 

Assessments  regarding  when  the  performance  measurement  baseline 
should  be  established. 

Experience  regarding  when  the  Demonstration  Review  or  Subsequent 
Application  Review  (SAR)  was  held. 

5. 3  Results 

5.3.1  Time  Actually  Used  to  Establish  the  C/SCSC-Related  Performance 
Measurement  baseline 


Government  and  contractor  business  managers  were  askeo  to  approximate  the 
number  of  months  after  contract  award  that  the  C/SCSC  performance  measurement 
baseline  was  established  (GBM-10  and  CBM-10).  The  results  are  shown  in  Table 
5.1. 
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TABLE  5.1 


riME  TO  establish  the  performance  measurement  baseline 


Number 

Government  Bus. 

Mqrs. 

y^mmsEssmasm 

KjEBHjj 

of 

Months 

Demonstration 

SAR 

Demonstration 

SAR 

At 

Least 

Less 

Than 

% 

% 

% 

% 

3 

37 

31 

15 

30 

3 

6 

31 

37 

37 

46 

6 

9 

8 

15 

18 

14 

9 

12 

8 

9 

6 

4 

12 

15 

4 

3 

18 

3 

15 

12 

5 

6 

3 

TUTAL 

1U0 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  who 
indicated,  in  number  of  months  after  contract  award,  when  the  performance 
measurement  baseline  was  established.  The  data  is  differentiated  by  the  type 
of  review. 


For  government  business  managers,  the  pattern  of  data  did  not  show  a  readily 
apparent  difference  between  contractors  undergoing  SARs  and  demonstration 
processes.  For  example,  the  government  business  managers  indicated  that  68% 
of  their  contracts  had  baselines  established  within  six  (6)  months, 
regardless  of  the  review  process. 

However,  based  on  the  contractor  respondent  data,  it  appears  to  take  longer 
to  establish  the  performance  measurement  baseline  for  contracts  undergoing 
the  demonstration  process  than  for  contracts  undergoing  the  SAR  process.  For 
example,  contractor  business  managers  indicated  that  baselines  were 
established  within  six  (6)  months  for  76%  of  the  contracts  undergoing  SaKs, 
but  for  only  52%  of  the  contracts  undergoing  the  demonstration  process. 

Since  contractor  business  managers  are  normally  more  closely  involved  with 
establishing  the  internal  baselines  on  their  contracts  than  are  the 
government  business  managers,  we  suspect  that  the  contractor  responses  are 
more  accurate  for  this  particular  question. 
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In  general,  the  result  obtained  from  the  contractor  respondents  should  be  the 
one  expected.  Contractors  undergoing  SARs  would  have  used  their  systems  on 
earlier  contracts  and  would  have  a  proven  capability  for  developing  a 
baseline.  Contractors  undergoing  the  demonstration  process  are  not  as  likely 
to  have  this  proven  capability. 

5.3.2  Effect  of  Undefinitized  Contracts  on  Establishing  the  Performance 
Measurement  Baseline 

Government  and  contractor  business  managers  were  asked  whether  their  contract 
was  definitized  at  the  time  of  contract  award  (GBM-8  and  CBM-8).  These 
responses  were  cross-tabul aced  with  the  responses  for  when  the  performance 
measurement  baseline  was  established  (GBM-10  and  CBM- 10).  The  results  are 
shown  in  Table  5.2. 


table  5.2 

EFFECT  OF  UNDEFINITIZED  CONTRACTS 


Number 

of 

Contract 

Definitized 

at  Contract 

Award 

Months 

Government  Bus. 

.  Mqrs. 

Contractor 

Bus.  Mqrs. 

At 

Less 

Yes 

No 

Yes 

No 

Least 

Than 

X 

% 

X 

% 

- 

3 

34 

28 

30 

20 

3 

6 

35 

33 

52 

2b 

6 

9 

11 

15 

13 

20 

9 

12 

8 

13 

3 

8 

12 

- 

12 

11 

2 

27 

TOTAL 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  who 
indicated,  in  number  of  months  after  contract  award,  when  the  performance 
measurement  baseline  was  established. 


Based  on  the  contractor  respondent  data,  it  appears  to  take  longer  to 
establish  the  performance  measurement  baseline  for  undefinitized  contracts 
than  for  contracts  that  are  definitized  when  awarded.  For  example, 
contractor  business  managers  indicated  that  baselines  were  established  within 


III  -  39 


six  (6)  months  for  82%  of  the  contracts  definitized  at  contract  award,  but 
for  only  45%  of  the  undef initized  contracts. 


As  in  the  data  which  differentiated  between  contracts  undergoing  the 
demonstration  or  SAR  process  (Table  5.1),  this  result  is  no:  as  apparent  from 
the  government  respondents.  For  example,  government  business  managers 
indicated  that  baselines  were  established  within  six  (6)  months  for  69%  of 
the  contracts  definitized  at  contract  award  and  for  61%  of  the  undef initized 
contracts . 

For  contracts  not  definitized  at  the  time  of  contract  award,  government  and 
contractor  business  managers  also  were  asked  to  approximate  how  many  months 
elapsed  from  contract  award  to  contract  def initization  (G8M-9  and  C8M-9) . 
These  responses  were  cross-tabulated  with  the  responses  for  when  the 
performance  measurement  baseline. was  established  (GBM-10  and  CBM-10).  The 
results  of  these  cross-tabulations  are  shown  in  Table  5.3. 


TABLE  5.3 

PERFORMANCE  MEASUREMENT  BASELINE  VERSUS  DEF I N IT IZAT ION  DATE 


Performance  Measurement 

Baseline  Established 

Government 

Bus.  Mqr. 

-  % 

Contractor 
Bus.  Mqr. 
% 

Earlier  than  when  Contract 
Definitized 

40 

46 

During  Same  Period  as  Contract 
Definitized 

33 

33 

Later  than  when  Contract 
Definitized 

27 

21 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  who 
indicated  that  the  C/SCSC-related  performance  measurement  baseline  was 
established  earlier,  later,  or  in  the  same  3-month  time  period  as  when 
the  contract  was  definitized. 


In  most  cases,  for  contracts  not  definitized  at  contract  award,  contractors 
establish  the  performance  measurement  baseline  earlier  or  in  the  same  time 
frame  that  the  contract  becomes  definitized.  It  appears  that  less  than  30% 
of  the  baselines  are  established  after  the  contract  is  definitized. 


Based  on  the  data  in  Table  5.2,  it  appears  that  for  undefinitized  contracts, 
the  performance  measurement  baseline  takes  longer  to  establish  than  for  other 
contracts.  However,  from  Table  5.3,  baselines  are  frequently  established 
prior  to  def initization  (40%  of  the  occurrences  according  to  government 
respondents  and  46%  according  to  contractor  respondents). 

5. 3. 3  Earliest  Reasonable  Time  to  Establish  the  C/SCSC-Related  Performance 
Measurement  Baseline 


Government  business  managers  were  asked  to  approximate  the  number  of  months 
after  contract  award  that  the  C/SCSC  performance  measurement  baseline  could 
reasonably  have  been  expected  to  be  established  (6BM-11).  The  results  are 
shown  in  Table  5.4. 


table  5.4 

EARLIEST  REASONABLE  TIME  THAT  BASELINE  COULD  HAVE  BEEN  ESTABLISHED 


Number 

of 

Months 

Government  Business 
Demonstration 

Managers 

SAk 

At  Least 

Less  Than 

% 

% 

- 

3 

35 

36 

3 

6 

37 

45 

6 

9 

14 

10 

9 

12 

4 

6 

12 

- 

10 

3 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of 
government  business  manager  resondents  who  indicated,  in  number 
of  months  after  contract  award,  when  the  performance  measurement 
baseline  could  reasonably  have  been  established.  The  data  are 
differentiated  by  the  type  of  review. 


Most  government  business  managers  apparently  expect  their  contractors  can 
establish  their  performance  measurement  baselines  within  six  (6)  months  of 
contract  award. 


Although  the  difference  appears  somewhat  small,  the  data  may  also  suggest 
that  government  business  managers  expect  contractors  undergoing  the  demonstra¬ 
tion  process  to  need  more  time  to  establish  their  baselines  than  contractors 
undergoing  the  SAR  process.  As  discussed  earlier,  contractors  undergoing  SAks 
already  would  have  a  proven  capability  in  establishing  baselines. 

To  get  a  clearer  insight  into  whether  government  business  managers  were 
satisfied  with  the  length  of  time  contractors  were  using  for  establishing 
their  performance  measurement  baselines,  we  cross-tabulated  these  responses 
(Table  5.4)  with  the  earlier  government  business  manager  responses  regarding 
when  the  baselines  were  actually  established  (Table  5.1).  These  results  are 
shown  in  Table  5.5. 


table  5.5 

EVALUATION  OF  BASELINt  TIMELINESS 


Government  Business  Managers 


Number  of 

Months 

Reasonable  Time  that 

Basel  ini 

to  Establish 

Basel ine 

Could 

Have 

Been  Established 

At 

Less 

Less 

Same 

More 

Total 

Least 

Than 

% 

% 

% 

- 

3 

0 

78 

22 

100 

3 

6 

19 

73 

8 

100 

6 

12 

57 

40 

3 

100 

12 

- 

69 

31 

0 

100 

The  numerical  values  in  this  table  indicate  the  percent  of  respondents 
who  felt  that  the  contract  performance  measurement  baseline  could 
reasonably  have  been  established  in  "less  "  time,  in  the  "same"  time, 
or  in  "more"  time  than  it  actually  was  established.  These  data  are  a 
cross-tabulation  of  data  used  as  the  sources  for  Table  5.1  (Business 
Manager)  and  Table  5.4. 


For  contractors  who  take  at  least  three  (3)  months  but  less  than  six  (6) 
months  to  establish  their  performance  measurement  baselines,  approximately 
one-fifth  (19%)  of  the  government  business  managers  apparently  feel  that  the 
job  can  be  done  more  quickly.  When  contractors  take  six  (6)  months  or  longer, 
then  most  of  these  managers  apparently  feel  that  the  baseline  can  be 
established  more  quickly. 
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5.3.4  Elapsed  Time  Until  Conduct  of  C/SCSC  Review 

Government  and  contractor  business  managers  were  askea  to  approximate  tne 
number  of  months  after  contract  award  that  the  Demonstration  Review  or 
Subsequent  Application  Review  (SAR)  was  held.  The  respondents  were  asked  to 
use  as  a  review  reference  only  the  one  visit  that  they  considered  the  key 
C/SCSC  review  for  this  contract  (GBM-12A  and  CBH-11A).  The  responses  are 
shown  in  Table  5.6. 


Table  5.6 

ELAPSED  TIME  TU  KEY  C/SCSC  VISIT 


Number 

of 

Months 

Governmemt  Bus.  Mqrs. 

Contractor  Bus 

:.  Mgr s . 

Demonstration 

SAR 

Demonstration 

SAR 

Least 

Less 

Than 

% 

% 

% 

% 

- 

3 

9 

3 

0 

2 

3 

6 

17 

33 

3 

28 

6 

12 

43 

37 

33 

44 

12 

18 

13 

13 

31 

12 

18 

24 

2 

6 

15 

6 

24 

30 

7 

3 

9 

4 

30 

- 

9 

0 

9 

4 

TOTAL 

100 

100 

lOo 

100 

The  numerical  values  in 

1  this  table  show 

the  percent  of 

:  respondents  who 

indicated,  in  number  of  months  after  contract  award,  when  the  one  key  C/sCsC 
visit  was  held.  The  data  is  differentiated  by  the  type  of  review. 


From  the  data,  the  percentage  of  occurrences  falling  within  ninety  (90) 
calendar  days  after  contract  award  (as  "suggested"  by  the  umr  clause  for 
C/SCSC)  was  nearly  negligiDle. 
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It  appears  from  the  data  that  what  was  regarded  as  the  key  visit  for  most  SARs 
occurred  within  the  first  year  of  the  contract  173%  according  to  government 
respondents  and  74%  according  to  contractor  respondents).  The  key  visit  for 
most  demonstrations  appears  to  have  occurred  within  eighteen  (18)  months  of 
contract  award  (82%  according  to  government  respondents  and  67%  according  to 
contractor  respondents). 

As  in  the  data  shown  earlier  in  Tables  5.1  and  5.2,  differences  in  Table  5.6 
between  contracts  undergoing  the  demonstration  process  versus  the  ShR  process 
are  more  apparent  for  contractor  respondents  than  for  government  respondents. 
An  additional  difficulty  related  to  the  data  specific  to  Table  5.6  is  the 
potential  difference  between  the  two  categories  of  respondents  regarding  which 
visit  might  be  perceived  as  the  one  key  visit,  particularly  for  the 
demonstration  process. 

Related  to  this  difficulty,  we  cross-tabulated  the  data  regarding  when  the 
performance  measurement  baseline  was  established  and  when  the  C/SCSC  visit, 
regarded  as  key,  was  held.  The  results  are  shown  in  Table  5.7. 


TABLE  5.7 


PERFORMANCE  MEASUREMENT  BASELINE  VERSUS  REVIEW  DATE 


Performance  Measurement 


Government 
Bus.  Mgr. 


Contractor 
Bus.  Mgr. 


Basel ine  Established 

Demo . 

% 

SAK 

% 

Demo . 

% 

Sar 

% 

Earlier  Than  Key  Review 

63 

66 

82 

70 

During  Same  Period  as  Key  Review 

22 

21 

12 

29 

Later  Than  Key  Review 

15 

13 

6 

1 

TUTAL 

1U0 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  who 
indicated  that  the  C/SCSC-related  performance  measurement  baseline  was 
established  earlier,  later,  or  in  the  same  time  period  as  when  the  key 
C/SCSC  visit  was  held. 


As  indicated  earlier  (paragraph  5.1)  it  would  be  difficult  to  have  an 
effective  demonstration  and  verification  process  prior  to  the  performance 
measurement  baseline  being  established.  However  from  these  data,  it  appears 
that  a  significant  number  of  government  business  managers  (15%  for  contracts 


undergoing  the  demonstration  process  and  13%  for  the  SAR  process)  indicated 
that  the  baseline  was  established  after  the  key  C/SCSC  visit.  It  may  be  that 
this  ke^  visit  was  not  uniformly  interpreted  (e . g - ,  Implementation  Visit  or 
Readiness  Assessment,  rather  than  the  Demonstration  Review). 

For  contractor  respondents,  there  also  was  a  difficulty  in  interpreting  data 
regarding  when  specific  events  (relating  to  the  review  process)  occurred. 
Contractor  business  managers  were  asked  to  approximate  the  number  of  months 
after  contract  award  that  they  received  official  notification  from  their 
contracting  office  that  their  system  had  been  sat isfactor i ly  applied  to  the 
contract  (C8M-14).  The  responses  are  shown  in  Table  5.8. 


TABLE  5.8 

ELAPSED  TIME  UNTIL  NOTIFICATION  OF  ACCEPTANCE 


Number  of  Months  Contractor  Business  Manager 


At  Least 

Less  Than 

Demonstration 

% 

SAR 

~ir 

- 

6 

18 

18 

6 

12 

3 

44 

12 

18 

18 

21 

18 

24 

29 

10 

24 

30 

7 

2 

30 

36 

3 

2 

36 

42 

18 

2 

42 

48 

0 

1 

48 

- 

4 

0 

TOTAL  100  100 


The  numerical  values  in  this  table  show  the  percent  of 
contractor  business  managers  who  indicated,  in  number  of 
months  after  contract  award,  when  they  received  official 
notification  that  their  system  had  been  sat isfactor i ly 
applied  to  this  contract. 


It  appears  from  the  data  that  official  notification  takes  longer  for  the 
demonstration  process  than  for  the  SAR  process.  For  example,  the  data  show 
that  within  twelve  (12)  months  after  contract  award,  orficial  notification  of 
satisfactory  application  was  received  in  21%  of  the  occurrences  for 
demonstrations  and  62%  for  SARs.  Alternatively,  the  data  show  that  two  (2) 
years  after  contract  award,  one-third  (32%)  of  these  notifications  remained 
for  demonstrations  while  only  7%  remained  for  SARs.  To  those  familiar  with 
the  C/SCSC  review  processes,  these  results  might  not  appear  to  be  unexpected. 

However,  these  data  should  be  interpreted  cautiously.  These  contractor 
business  manager  responses  were  cross-tabulated  with  the  contractor  business 
manager  responses  for  when  the  C/SCSC  review  was  held  (Table  5.6).  The 
cross-tabulation  showed  several  occurrences  where  a  respondent  indicated  that 
official  notification  was  received  prior  to  when  the  same  respondent  indicated 
the  applicable  C/SCSC  review  was  held.  This  result  was  found  in  21%  of  the 
responses  for  demonstrations  and  13%  for  SARs;  including,  for  the 
demonstration  process,  four  (4)  out  of  five  (5)  of  the  notifications  which 
were  indicated  as  occurring  within  the  first  six  (6)  months  of  contract 
award.  To  our  knowledge,  these  results  are  not  in  accord  with  normal  practice 

5.4  Summary 

Based  on  the  respondent  data,  we  conclude  that: 


The  C/SCSC-related  performance  measurement  baseline  for  most 
contracts  is  established  within  six  (6)  months  of  contract  award. 
According  to  contractor  business  managers,  it  appears  to  take  longer 
to  establish  the  baselines  for  contracts  undergoing  the 
demonstration  process  than  for  contracts  undergoing  the  SAR 
process.  Since  contractors  undergoing  SARs  should  have  a  previously 
proven  capability  for  developing  a  baseline,  this  should  be  expected 


According  to  contractor  business  managers,  for  contracts  that  are 
not  definitized  at  contract  award,  it  appears  to  take  longer  to 
establish  the  performance  measurement  baseline  than  for  those 
contracts  that  were  definitized  at  contract  award.  It  also  appears 
that  somewhat  less  than  half  of  the  undef initized  contracts  have 
their  baselines  established  prior  to  def initization . 


Most  government  business  managers  appear  to  be  satisfied  that  the 
elapsed  time  is  reasonable  when  performance  measurement  baselines 
are  established  within  six  (6)  months  of  contract  award.  Most 
government  business  managers  also  appear  to  feel  that  when  baselines 
take  longer  than  six  (6)  months,  it  could  have  been  done  more 
quickly. 

A  negligible  number  of  C/SCSC  reviews  appear  to  fall  within  the 
ninety  (90)  calendar  day  time-frame  contained  within  the  DAR  C/SCSC 
clause.  Most  C/SCSC  visits  regarded  as  key  by  the  respondents 
appeared  to  occur  within  eighteen  (18)  months  of  contract  award  for 
the  demonstration  process  and  within  twelve  (12)  months  of  contract 
award  for  the  SAR  process. 
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6.0 


COST /SCHEDULE  PERFORMANCE  REPORTING 


6. 1  Background 

Depth,  timeliness  and  use  of  cost/schedule  data  have  been  continuing  concerns. 

General  guidance  regarding  depth  and  timeliness  is  contained  in  the  Data  Item 
Description  for  the  Cost  Performance  Report  (CPR).  The  guidance  indicates 
that  the  level  of  detail  to  be  reported  in  the  CPR  will  normally  be  limited  to 
level  three  (3)  of  the  contract  work  breakdown  structure  (CWBS).  For 
timeliness,  the  guidance  indicates  that  the  CPR  should  be  submitted  to  the 
government  no  later  than  twenty-five  (25)  calendar  days  following  the 
reporting  cutoff  date.  Within  this  general  guidance,  each  program  office  (as 
the  intended  CPR  user)  is  essentially  responsible  for  establishing  its  own 
reporting  requirements. 

In  this  area,  we  assessed  the  depth,  timeliness  and  use  of  cost/schedule 
performance  data.  We  included  assessments  related  to  the  summary  level  CPR 
data  as  well  as  the  lower  level  internal  contractor  data. 


6.2  Overview  of  Related  Questions 


We  included  questions  which  directly  adaressed  assessments  and  perceptions  in 
the  six  (6)  areas  listed  below.  The  respondents  for  these  questions  were 
program  managers  and  business  managers,  both  government  and  contractor. 


Assessments 

data. 

Assessments 

Assessments 
submi ttal s . 

Assessments 

Assessments 

Assessments 

data. 


and  perceptions  regarding  the  depth  and  volume  of  CPR 

regarding  the  volume  of  internal  contractor  reports, 
and  perceptions  regarding  the  timeliness  of  CPR 

regarding  the  timeliness  of  internal  contractor  reports. 

and  perceptions  regarding  the  use  of  CPRs. 

regarding  the  use  of  contractor  internal  cost/schedule 


6.3  Results 


6.3.1  Depth  of  Cost  Performance  Report  Data 

Government  and  contractor  business  managers  were  asked  to  indicate  the  lowest 
CWBS  level  regularly  reported  on  the  CPR  (GbM-16  and  CBM-16).  The  results  are 
shown  in  Table  6.1 . 
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TABLE  6.1 


CWBS  REPORTING  LEVEL 


Lowest  CWBS 

Level  Regularly 

Reported  on  CPR 

Government 
Bus.  Mqr. 

% 

Contractor 
Bus.  Mqr. 

'1 

2 

5 

7 

3 

64 

50 

4 

19 

24 

5 

8 

17 

6  or  lower 

4 

2 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of 
respondents  that  indicated  the  CWBS  level  shown,  as  the 
lowest  level  regularly  reported  on  the  CPR. 


The  majority  of  respondents  indicated  that  CWBS  level  3  or  higher  was  the 
lowest  level  regularly  reported  (i.e.,  for  government  respondents  and  57% 
for  contractor  respondents).  However,  many  respondents  indicated  lower  level 
reporting.  Less  than  5%  of  the  respondents  indicated  reporting  below  CWBS 
level  5. 

Based  on  cross-tablulations  of  the  respondent  data,  there  appeared  to  be  some 
differences  in  CWBS  reporting  levels  related  to  the  type  of  contract  work 
(development  versus  production),  contract  value,  contract  type  and  weapons 
system.  Some  of  these  results  are  shown  in  Tables  b.2,  6.3  and  6.4. 
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TABLE  6.2 


DEVELOPMENT  VERSUS  PRODUCTION  WORK 


CWBS  Level  3  or 

Government 

Contractor 

Higher 

Bus.  Mqr. 

Bus.  Mqr. 

Type  of  Work 

% 

i 

Development 

67 

47 

Low  Rate  Production 

73 

67 

Production 

81 

70 

Overal  1 

69 

57 

The  numerical  values  in  this  table  show  the  percent  of 
respondents  who  indicated  the  lowest  regular  reporting  level 
was  CwBS  level  3  or  higher,  for  the  type  of  work  shown. 


TABLE  6.3 
CONTRACT  VALUE 


CWBS  Level  3  or 

Higher _ 

_ Contract  Value 

Less  Than  1>160  Million 

$160  Million  or  More 

Overal 1 


Gov ernment 

Contractor 

bus.  Mqr. 

Bus.  Mqr. 

i 

% 

75 

b2 

63 

52 

69 

57 

The  numerical  values  in  this  table  show  the  percent  of 
respondents  who  indicated  the  lowest  regular  reporting  level 
was  CW8S  level  3  or  higher,  for  the  contract  value  shown. 


TABLE  6.4 


COST  PLUS  VERSUS  FIXED  PRICE  INCENTIVE  CONTRACTS 


CWBS  Level  3  or 

Higher 

Government 
Bus.  Mqr. 

Contractor 
Bus.  Mqr. 

Contract  Type 

% 

% 

Cost  Plus 

b5 

55 

Fixed  Price  Incentive 

75 

62 

Overall 

69 

57 

The  numerical  values  in  this  table  show  the  percent  of 
respondents  who  indicated  the  lowest  regular  reporting  level 
was  CWBS  level  3  or  higher,  for  the  contract  type  shown. 


Both  categories  of  respondents  in  Table  6.2  indicate  a  higher  percentage  of 
the  production  contracts  being  reported  at  CWBS  level  3  or  higher.  This 
pattern  of  data  suggests  that  CWBS  reporting  levels  are  "higher"  on  production 
contracts  than  on  development  contracts.  This  result  would  appear  to  be 
consistent  with  guidance  contained  in  the  DOD  C/SCSC  Joint  Implementation 
Guide  regarding  differences  between  cost  account  levels  on  development  and 
production  type  work. 

The  pattern  of  data  in  Table  6.3  suggests  that  CWBS  reporting  levels  are  also 
"higher"  on  contracts  valued  at  less  than  $160  million  than  on  contracts 
valued  at  £160  million  or  more  (i.e.,  both  categories  of  respondents  show  a 
higher  percentage  of  contracts  valued  at  less  than  $160  million  being  reported 
at  CWBS  level  3  or  higher).  This  result  appears  reasonable  since  higher 
dollar  value  contracts  could  have  significantly  higher  dollar  value  CWBS 
reporting  elements,  without  the  additional  lower  level  reporting. 

In  Table  6.4,  both  categories  of  respondents  indicate  a  higher  percentage  of 
fixed-price  incentive  contracts  being  reported  at  CWBS  level  3  or  higher,  thus 
suggesting  lower  CWBS  reporting  levels  on  cost-type  contracts.  This  result 
also  appears  reasonable  since  cost-type  contracts  normally  entail  higher  risk 
than  fixed-price  incentive  contracts.  With  higher  risk,  we  could  expect  a 
customer  requirement  for  increased  visibility. 

The  pattern  of  data  in  an  additional  cross-tabulation  with  weapon  system 
category  suggested  that  electronics  contracts  had  lower  CWBS  reporting  levels 
than  did  contracts  for  other  types  of  weapon  systems.  These  data  are  not 
shown. 


Government  and  contractor  business  managers  also  were  asked  their  opinions 
regarding  whether  the  CW8S  reporting  levels  on  their  contracts  were 
appropriate.  Government  business  managers  were  asked  whether  the  CWBS  level 
was  appropriate  "for  meaningful  contract  analysis  (GBM-17)".  Contractor 
business  managers  were  asked  whether  the  "depth"  of  reporting  was  appropriate 
( CBM -17).  The  responses  are  shown  in  Table  6.5. 


TABLE  6.5 


OPINION  REGARDING 

CWBS  REPORTING 

LEVEL 

CWBS  Reporting 

Government 

Contractor 

Level  Oepth 

Bus.  Mqr. 
% 

Bus.  Mqr. 
-  T  ' 

Too  Low 

6 

21 

Just  About  Right 

83 

77 

Too  High 

11 

2 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents 
voicing  the  opinion  indicated.  Government  business  managers  were 
asked  their  opinion  with  reference  to  the  CWBS  level  needed  for 
“meaningful  contract  analysis".  Contractor  business  managers  were 
asked  their  opinion  solely  with  reference  to  the  reporting  level 
"depth"  within  the  CWBS.  For  both  categories  of  respondents,  those 
who  selected  "No  opinion"  are  not  included  in  this  table. 


Most  respondents  felt  that  the  CWBS  reporting  levels  were  "Just  about  right" 
(83%  for  government  respondents  and  77%  for  contractor  respondents,  as  shown). 

However,  differences  in  needs  and  perceptions  between  the  government  and 
contractor  respondents  are  apparent  in  the  percentage  differences  for  the 
responses  of  "Too  low"  and  "Too  high". 

Cross-tabulations  (GBM-16  and  17)  (not  shown)  indicated  that  for  the  11%  of 
the  government  business  managers  who  indicated  the  CWBS  level  was  "Too  high" 
for  meaningful  analysis,  88%  indicated  their  CWBS  reporting  level  as  level  3 
or  higher.  The  remaining  12%  were  at  CWBS  level  4.  For  the  21%  of  contractor 
business  managers  who  indicated  that  the  depth  of  reporting  was  "Too  low", 
cross-tabulations  (CBM-16  and  17)  inot  shown)  showed  that  96%  indicated  their 
CWBS  reporting  level  as  level  4  or  lower.  The  remaining  4%  were  at  CWBS  level 
3. 
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To  further  explore  perceptions  of  potentially  excessive  reporting,  contractor 
business  managers  were  asked  to  indicate  the  extent  to  which  they  believed  the 
requested  data  was  excessive  (extremely  excessive,  somewhat  excessive  or  not 
excessive)  for  each  CPR  format  (CBM-19).  Their  responses  are  shown  in  Table 
6.6. 


table  6.6 

CONTRACTOR  PERCEPTIONS  OK  CPR  DATA  EXCESSIVE  To  DOD  NEEDS 


Extent  of  CPR  formats 


Data  Being 

Contract 

Funct  '1 

Baseline 

Manpower 

Problem 

Reported 

MBS 

jESHSm; 

Analysis 

% 

% 

% 

% 

% 

Excessive 

27 

18 

20 

23 

44 

Not  Excessive 

73 

82 

80 

77 

56 

TUlrtL 

100 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  contractor  business 
manager  respondents  who  indicated  that  they  believed  the  data  being  requested 
for  each  format  was  excessive  or  not  excessive.  The  responses  showing 
"excessive"  are  for  combined  evaluations  of  extremely  excessive  and  somewhat 
excessive. 


It  is  apparent  from  the  data  that  most  contractor  business  managers  do  not 
believe  that  the  CPR  data  being  submitted  is  excessive  to  the  DOD  program 
office's  needs.  However,  it  is  also  apparent  that  more  respondents  feel  the 
Problem  Analysis  format,  more  than  other  formats,  contains  excessive  data. 

From  cross-tabulations  (not  shown),  a  majority  of  the  respondents  (53%)  feel 
that  Problem  Analysis  is  excessive  on  development  type  contract  while  only 
33%  feel  that  Problem  Analysis  is  excessive  on  production  contra  s. 

Regarding  government  business  manager  assessment  of  their  own  ne>  js,  these 
managers  were  asked  whether  the  number  of  CPR  pages  (not  solely  submitted  at 
the  contractor's  choice)  could  be  reduced  without  hampering  the  government 
program  of f ice' s  ability  to  manage  the  contract  (GBM-19  and  20).  Seventeen 
percent  (17%)  indicated  that  the  number  of  pages  could  be  reduced. 

Thirty-three  percent  (33%)  of  this  17%  (or  less  than  6%  of  the  original  number 
of  respondents)  indicated  that  more  than  20%  of  the  pages  could  be  reduced 
( &BM-21 ) . 
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Based  on  cross-tabulations  (GBM-17  and  1 (not  shown),  only  10%  of  all 
government  business  managers  who  iniic  .ted  that  CPK  pages  could  be  reduced 
also  indicated  that  their  CwBS  reporting  level  was  "Too  low".  This  appears  to 
suggest  that  the  "excess"  pages  are  not  necessarily  related  to  CWBS  reporting 
levels. 

from  these  government  business  manager  data,  it  appears  that  most  of  tne  cPk 
information  is  felt  to  be  needed  by  its  recipients.  Since  reduction  of  what 
is  not  needed  would  appear  to  be  under  the  control  of  the  governnent  program 
office,  it  is  not  clear  why  the  apparently  unneeded  information  is  being 
received.  Possible  explanations  include;  different  views  by  otner  people  in 
the  program  office  regarding  the  need  for  the  information;  no  evident  cost 
savings;  lower  priority  for  getting  the  information  reduceu  than  for  other 
things  to  be  done;  and  the  possibility  that  such  information  might  he  needeu 
in  the  future. 

6.3.2  Volume  of  Internal  Contractor  Reports 

Contractor  business  managers  were  asked  whether  the  number  of  pages  of 
internal  reports  generated  each  month  from  their  C/SCSC-accepted  system  toulo 
be  reduced  without  hampering  their  ability  to  manage  the  contract  (LBM-20). 
Those  respondents  who  indicated  that  the  number  of  pages  could  be  reduceo  wert 
further  asked  to  give  a  rough  estimate  of  the  percentage  of  pages  that  could 
be  eliminated  (CBM-21).  The  combined  responses  (CBM  20  and  21)  are  shown  in 
Table  6.7. 


TrtBLE  6,7 

INTERNAL  PmGCB  THAI  COULD  BE  REDUCED 


Internal  Report  Contractor 

Pages  That  Could  be  Bus.  Mgr. 

El iminated 

_ % _  % 

None  35 

1-10  10 

11-20  20 

21-30  22 

31  or  More  13 

TOTAL  100 


The  numerical  values  in  this  table  show  the  percent  of 
contractor  business  managers  that  indicated  the  percent  of 
internal  report  pages  as  shown,  that  could  be  eliminated 
without  hampering  their  ability  to  manage  the  contract. 
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Most  contractor  business  managers  (65%)  feel  that  some  reductions  can  be  made 
in  the  number  of  pages  of  internally  generated  reports  from  their 
C/SCSC-accepted  systems  without  hampering  their  ability  to  manage  their 
contracts.  Included  within  this  majority  are  35%  who  feel  that  the  number  of 
pages  can  be  reduced  by  21%  or  more.  This  could  represent  a  substantial 
volume  of  internal  reports. 

Contractor  business  manager  responses  regarding  excess  internal  reports  were 
cross-tabulated  with  contractor  business  manager  perceptions  of  whether  the 
benefits  of  C/SCSC  outweighed  its  costs  (Table  1.4),  and  with  contractor 
business  manager  approximations  of  the  cost  of  system  operation  (Table  1.8). 

Overall,  55%  of  the  contractor  business  managers  agreed  that  the  benefits  of 
C/SCSC  outweighed  its  costs  (as  shown  in  Table  1.4).  However  for  those  who 
indicated  that  none  of  their  internal  report  pages  could  be  reduced,  7u% 
agreed  that  the  benefits  of  C/SCSC  outweighed  its  costs.  For  those  who 
indicated  that  1-20%  of  the  pages  could  be  eliminated,  64%  agreed  that  the 
benefits  outweighed  the  costs;  but  for  those  who  indicated  that  21%  or  more  of 
their  pages  could  be  eliminated,  only  31%  agreed  that  the  benefits  outweighed 
the  costs.  The  pattern  of  these  data  suggests  that  contractor  perceptions  of 
the  "benefit  versus  cost"  of  C/SCSC  are  related  to  assessments  of  excessive 
internal  reports,  with  particularly  unfavorable  impact  when  the  estimate  of 
excessive  pages  is  21%  or  more. 

Cross-tabulations  with  the  cost  of  system  operation  showed  very  little 
apparent  difference  in  cost  between  respondents  who  indicated  that  pages  could 
be  reduced  and  respondents  who  indicated  that  pages  coula  not  be  reduced 
(C6M-20).  However,  for  respondents  who  indicated  that  pages  could  be  reduced 
( CBM-21 ) ,  the  data  suggests  apparent  differences  between  those  who  assess  the 
volume  of  excess  pages  as  1-20%  and  those  who  assess  the  volume  of  excess 
pages  as  21%  or  more.  For  example,  the  majority  of  contractor  business 
managers  who  indicated  that  1-20%  of  their  pages  could  be  eliminated  ( 5o% ) 
approximated  the  cost  of  operation  as  10  man-years  per  year  or  less.  The 
majority  of  contractor  business  managers  who  indicated  that  21%  or  more  of 
their  pages  could  be  eliminated  (62%)  approximated  the  cost  of  operation  as  11 
man-years  per  year  or  more.  The  data  appears  to  suggest  that  where  the 
estimated  number  of  internal  report  pac^.,  that  could  be  eliminated  is  21%  or 
more,  the  approximated  number  of  man-years  per  year  spent  on  operating  the 
C/SCSC-accepted  system  is  also  higher. 

Contractor  business  manager  opinions  regarding  excess  internal  reports  were 
also  tabulated  against  their  perceptions  of  excess  CPR  data  being  provided  to 
the  UuL)  program  offices  (CBM-19  and  20).  The  results  of  these  tabulations  are 
shown  in  Table  6.8. 
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TABLE  6.8 


EXCESS  INltRWHL  VERSUS  EXTERNAL  UATA 


Believe  that 
CPR  Data  is  Excessive 


internal  Report  Pages  Could  be 
Reduced  Without  Hampering 
Contractor's  Ability  to  Manage 


CPR  Formats 

Yes 

% 

No 

_% 

Overal 1 
% 

Contract  WBS 

32 

19 

27 

Functional  Organization 

23 

11 

18 

Baseline 

26 

7 

20 

Manpower 

28 

14 

23 

Problem  Analysis 

51 

29 

44 

The  numerical  values  in  this  table  indicate  the  percent  of 
contractor  business  manager  respondents  who  indicated  that 
they  believed  the  CPR  data  requested  were  excessive. 


The  pattern  of  data  suggests  that  for  contractors  who  feel  their  internal 
report  pages  can  be  reduced  without  hampering  their  ability  to  manage,  there 
is  a  higher  percentage  (than  for  those  who  do  not  feel  their  internal  reports 
can  be  reduced)  who  also  feel  that  CPR  data  is  excessive  to  government  needs. 
For  example,  51%  of  those  who  believe  that  their  internal  reports  can  be 
reduced  also  feel  that  the  CPR  Problem  Analysis  format  contains  data  which  is 
excessive  to  government  needs.  By  comparison,  only  29%  of  the  contractors  who 
believe  their  internal  reports  cannot  be  reduced  feel  the  CPR  Problem  Analysis 
format  contains  data  which  is  excessive  to  government  needs. 

In  general,  the  data  appears  to  suggest  that  contractors'  perceptions  of 
government  needs  for  CPR  data  are  related  to  whether  the  contractors  believe 
they  are  generating  internal  reports  in  excess  of  their  own  needs. 

6.3.3  Timeliness  of  Cost  Performance  Reports  (CPRs) 

Government  business  managers  were  asked  approximately  how  many  calender  weeks 
after  the  close  of  the  contractor's  reporting  period  are  CPRs  received 
(GBM-22).  Similarly,  contractor  business  managers  were  asked  approximately 
how  many  calendar  weeks  after  the  close  of  their  reporting  period  do  they 
forward  their  CPRs  to  the  government  iCBM-25).  The  responses  are  shown  in 
Table  6.y. 
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TABLE  6.9 


|  - 

!  CPR  TRANSMITTAL  DATE 


Calendar  Weeks  After 

Close  of  Reporting  Period 

Government 

Bus.  Mgr. 
Received 

Contractor 
Bus.  Mgr. 
F  orwaraed 

At  Least 

Less  lhan 

% 

% 

- 

2 

5 

1 

2 

3 

7 

22 

3 

4 

36 

54 

4 

5 

32 

19 

5 

- 

20 

4 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents 
who  indicated,  in  number  of  calendar  weeks  after  the  reporting 
period  closeout,  when  the  CPR  was  forwarded  to  or  received  by  the 
government  program  office. 


Most  CPRs  are  forwarded  to  the  government  within  four  (4)  weeks  after  the 
close  of  the  contractor's  reporting  period  (77%).  Most  CPRs  are  received  by 
government  program  offices  within  five  (5)  weeks  after  the  close  of  the 
contractor's  reporting  period  (80%). 

These  responses  were  cross-tabulated  by  military  department.  (Note:  The 
military  departments  are  identified  in  the  following  tables  and  discussion  as 
Departments  A,  B,  and  C.  This  was  done  to  preclude  focusing  on  the  specific 
military  departments,  rather  than  the  conclusion  that  follows.  The 
association  of  the  specific  military  department  with  the  data  should  not 
affect  the  subsequent  conclusion.)  The  results  are  shown  for  government 
business  managers  in  Table  6.10.  (Results  by  military  department  for 
contractor  business  managers  reasonably  agreed  with  the  results  for  government 
respondents,  except  for  a  "forwarding"  lead-time  similar  to  that  indicated  in 
Table  6.9) . 


Ill  -  56 


TABLE  6.10 


C PR  RECEIPT  OATE  BY  MILITARY  DEPARTMENT 


Calendar  Weeks  After 


At  Least 

Less  lhan 

- 

2 

2 

3 

3 

4 

4 

5 

5 

- 

TOTAL 


Government  Business  Manager 
ABC 


~%~ 

~T 

- 

2 

9 

- 

10 

9 

28 

21 

51 

44 

30 

26 

2b 

37 

5 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of 
respondents  who  indicated,  in  number  of  calendar  weeks  after  the 
reporting  period  closeout,  when  the  CPR  was  received  by  the 
program  office.  A,  B,  and  C  represent  the  three  (3)  military 
departments. 


Based  on  the  data.  Department  C  programs  appear  to  receive  their  CPRs  earlier 
than  the  other  services,  r or  example,  69%  of  the  Department  C  CPRs  are 
received  within  four  (4)  weeks  while  only  approximately  30%  are  received  by 
the  other  services  within  the  same  time-period. 

Government  program  managers  and  business  managers  were  asked  if  they  agreed  or 
disagreed  with  the  statement  that  CPRs  are  received  on  a  sufficiently  timely 
basis  so  as  to  be  useful  to  the  program  office  (6PM-15  and  GBM-30).  The 
results  are  shown  by  military  department  in  Table  6.11.  The  data  shows  the 
percentage  of  respondents  who  agreed  with  the  statement. 
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TABLE  6.11 


ASSESSMENT  OF  CPR  TIMELINESS 


CPR  is  Sufficiently 

Timely  so  as  to  be  Useful 

Government 

Prog.  Mgr.  Bus.  Mgr. 

Military  Department 

% 

% 

A 

24 

51 

8 

38 

61 

C 

58 

67 

Overal  1 

43 

61 

The  numerical  values  in  this  table  indicate  the  percent  of 
respondents,  by  military  department,  that  agreed  with  the 
statement  that  the  CPR  is  sufficiently  timely. 


While  most  government  business  managers  appear  to  be  satisfied  with  CPR 
timeliness,  most  government  program  managers  are  not  satisfied.  There  also 
appears  to  be  differences  between  the  military  departments .  department  C 
managers  appear  to  be  more  satisfied  with  the  timeliness  of  CPRs  than  their 
counterparts  in  other  services.  The  difference  in  the  level  of  agreement 
between  the  Department  A  and  C  program  managers  is  statistically  significant 
at  the  957o  confidence  level. 

As  shown  earlier  from  Table  6.1U,  a  majority  of  Department  C  business  managers 
(69%)  indicated  that  they  received  their  CPRs  within  four  (4)  weeks.  From  the 
same  table,  it  takes  between  four  (4)  and  five  (5)  weeks  within  the  other 
services  to  achieve  the  same  approximate  percentage  (72%  for  Department  A  and 
63%  for  the  Department  B).  It  therefore  appears  that  CPRs  should  be  received 
within  four  (4)  weeks  for  program  managers  to  feel  that  CPRs  are  sufficiently 
timely. 

CPR  receipt  dates  were  cross-tabul ated  with  government  business  manager 
responses  regarding  whether  the  CPR  was  sufficiently  timely  to  be  useful  iGo^ 
22  and  3 u ) .  The  results  are  shown  in  Table  6.12. 
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TABLE  6.12 


CPR  TIMELINESS  VERSUS  CPR  RECEIPT  DATE 


Calendar  Weeks  After 

Close  of  Reporting  Period 

Government  Business  Managers 
Who  Agreed  That  CPR  was 
Sufficiently  Timely 

At  Least 

Less  Than 

% 

- 

2 

100 

2 

3 

70 

3 

4 

63 

4 

5 

55 

5 

- 

52 

Overal 1 

61 

The  numerical  values  in  this  table  show  the  percent  of  government 
business  managers  who  agreed  that  the  CPR  was  sufficiently  timely  to 
be  useful,  for  each  CPR  receipt  date  indicated. 


While  the  majority  of  government  business  managers  appear  to  feel  that  the  CPR 
is  sufficiently  timely  to  be  useful  over  the  entire  range  of  receipt  dates, 
there  is  much  higher  agreement  for  the  earlier  receipt  dates. 

6.3.4  Timeliness  of  Internal  Contractor  Cost/Schedule  Reports 

Contractor  business  managers  were  asked  approximately  how  many  weeks  after  the 
close  of  their  monthly  reporting  period  that  C/SCSC-related  reports  were 
provided  to  their  Cost  Account  Managers  (CBM-24).  The  responses  are  shown  in 
Table  6.13. 
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TABLE  6.13 


INTERNAL  REPORTS  TRANSMITTAL  DATE 


Calendar 

Weeks  After 

Close  of  Reporting  Period 

At  Least 

Less  Than 

- 

1 

1 

2 

2 

3 

3 

- 

TOTAL 


Contractor 

Business 

Manager 

Demonstration  SAR 

Overal 1 

% 

”  T" 

%  ' 

16 

28 

25 

56 

48 

50 

19 

15 

16 

9 

9 

9 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents 
who  indicated,  in  number  of  calendar  weeks  after  the  reporting 
period  closeout,  when  C/SCSC-related  reports  were  provided  to  their 
Cost  Account  Managers. 


Most  contractors  (75%)  provide  internal  C/SCSC-related  reports  to  their  cost 
account  managers  within  two  (2)  weeks  after  the  reporting  period  cutoff  date. 
The  percentages  of  contractors  who  provide  the  reports  within  the  first  week 
(16%  for  contractors  who  had  undergone  demonstrations  and  28%  for  contractors 
who  had  undergone  SARs)  appear  to  indicate  that  contractors  who  had  undergone 
demonstrations  take  longer  to  provide  the  reports  than  those  who  had  undergone 
SARs.  This  result  appears  reasonable,  since  contractors  who  had  undergone 
SARs  would  be  likely  to  have  had  more  experience  with  their  C/SCSC-accepted 
systems . 

Contractor  program  and  business  managers  were  asked  whether  they  agreed  that 
their  internal  cost  and  schedule  reports  were  received  on  a  sufficiently 
timely  basis  to  be  useful  to  them  (CPM-19  and  CBM-29).  Most  managers  in  both 
categories  agreed  that  their  internal  reports  were  timely  (72%  for  contractor 
program  managers  and  74%  for  contractor  business  managers). 

The  dates  that  internal  reports  were  provided  to  cost  accounts  managers  were 
cross-tabulated  with  contractor  business  manager  responses  regarding  whether 
the  reports  were  sufficiently  timely  to  be  useful  (CBM-24  and  29).  The 
results  are  shown  in  Table  6.14. 
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TrtBLt  6.14 


INTERNAL  REPORT  TIMELINESS  VERSUS  RECEIPT  DATE 


Calendar  Weeks  After 
Close  of  Reporting  Period 


At  Least  Less  Than 


Contractor  Business  Managers  Who 
Agreed  That  Internal  Cost/Scheaule 
Reports  Were  Sufficiently  Timely 
% 


1 


85 


1 


2 


7b 


2  3 


73 


3 


2b 


Overall 


74 


The  numerical  values  in  this  table  show  the  percent  of  contractor 
business  managers  who  agreed  that  internal  cost/schedule  reports  were 
sufficiently  timely  to  be  useful,  for  each  receipt  date  indicated. 


Tne  data  appear  to  suggest  that  most  contractor  business  managers  consider 
their  internal  cost/schedule  reports  sufficiently  timely  to  be  useful  when  the 
reports  are  provided  to  their  cost  account  managers  within  three  (3)  weeks 
after  the  close  of  the  reporting  period. 

6.3.5  Cost  Performance  Report  (CPR)  Use 

Government  business  managers  were  asked  to  what  extent  their  program  offices 
routinely  review  CPRs  each  month  (GBM-18).  The  selections  and  responses  for 
each  selection  are  shown  in  Table  6.15. 
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TABLt  6.15 


EXTENT  OF  ROUTINE  CPR  REVIEW 


bovernment 


Extent  of  CPR  Review  Bus.  Mgr. 

_ _  % 

Entire  Report  74 

Only  Certain  Formats  20 

Only  for  Exceeded  Thresholds  4 

Do  Not  Review  Report  2 

TOTAL  100 


The  numerical  values  in  this  table  show  the  percent  of 
government  business  managers  who  selected  the  choices 
shown,  for  the  extent  of  their  CPR  reviews. 


As  shown  in  the  data,  most  government  business  managers  indicate  that  their 
program  offices  review  the  CPR  each  month  in  its  entirety. 

Government  business  managers  also  were  asked  how  many  calendar  weeks  elapse 
from  the  time  CPRs  are  received  until  the  reviews  and  analyses  within  the 
program  office  are  completed  (bBM-23).  The  responses  are  shown  in  Table  6. lb. 


TABLE  6. 1 6 


ELAPSED  TIME  FROM  RECEIPT  OF  CPR  TO  COMPLETION  OF  CPR  REVIEW 


Calendar  Weeks 

To  Perform  CPR  Review 

Government 

Bus.  Mcjrs. 

At  Least 

Less  Than 

Analysis  Complete 
% 

- 

1 

18 

1 

2 

46 

2 

3 

21 

3 

4 

7 

4 

- 

8 

TOTAL 

100 

The  numerical  values  in  this  table  show  the  percent  of 
government  business  managers  who  indicated,  in  number 
of  calendar  weeks  after  receipt,  when  their  CPR  review 
and  analyses  were  completed. 


Most  government  business  managers  164%)  indicated  that  their  CPR  analysis  was 
completed  within  two  (2)  weeks  after  receipt  of  the  CPR. 

The  amount  of  time  to  complete  CPR  reviews  (Table  6.16)  was  cross-tabulated 
with  government  business  managers'  ratings  of  how  well  (excellent,  good,  fair, 
poor,  or  no  opinion)  the  CPR  helps  to  determine  cost  status  and 
estiinates-at-completion  (Table  1.3).  The  results  are  shown  in  Table  6.17. 
Percentages  are  shown  for  combined  ratings  of  "excellent"  and  "good". 


Ill  -  63 


TABLE  6.17 


TIME  TO  COMPLETt  CPR  REVIEW  VERSUS  RATING  OF  CPR 


Contract-Related 

Calendar  Weeks  to  Perform 

CPR  Review 

Status 

Less  Than 
% 

2  2  or  More 

% 

Overal 1 
% 

Cost  Status 

93 

6b 

83 

Estimates-at-Completion 

69 

60 

60 

The  numerical  values  in  this  table  show  the  percent  of  government 
business  managers  who  rated  the  CPR  as  "excellent"  or  "good"  in 
helping  to  determine  contract-related  status. 


As  might  be  anticipated,  for  program  offices  that  complete  their  CPR  reviews 
more  quickly  (in  less  than  2  weeks,  rather  than  2  weeks  or  more),  a  greater 
percentage  of  business  managers  apparently  rate  their  CPRs  as 
good-to-excel lent  (in  helping  to  determine  contract  status). 

To  approximate  the  amount  of  elapsed  time  that  completion  of  CPR  analysis 
represents  in  relation  to  the  cutoff  date  of  the  cost/schedule  information 
being  reviewed,  the  results  of  Table  6.1b  were  cross-tabulated  with  the  number 
of  calendar  weeks  after  the  contractor's  cutoff  date  that  the  CPR  was  received 
(Table  6.9).  The  results  of  this  tabulation  are  shown  in  Table  6.18.  (Note: 
The  scale  for  this  table  is  shown  in  two-week  increments,  and  the  increments 
overlap.  This  is  the  result  of  the  way  the  scales  are  constructed  in  the 
questions/responses  (taBM-22  and  GBM-23)  being  cross-tabulated.) 
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TABLt  6.18 


ELAPSED  TIME  FROM  CLOSE  OF  kfcPQRT  PERIOD  TU  COMPLETION  OF  IPR  RtVItW 


Calendar  Weeks  After 
Close  of  Contractor's 
Accounting  Period 

Government 

Bus.  Mgr. 
Analysis  Complete 

At  Least  Less  Than 

% 

- 

2 

1 

1 

3 

1 

2 

4 

3 

3 

5 

11 

4 

6 

21 

5 

- 

29 

6 

- 

22 

7 

- 

12 

TUTAL 

100 

The  numerical 

values  in 

this  table  show  the  percent 

of  government  business  managers  who  indicated,  in 
number  of  calendar  weeks,  when  CPRs  were  received 
and  when  the  review  and  analyses  were  completed. 


Most  program  offices  apparently  do  not  complete  their  OPR  reviews  until  after 
the  following  report  period's  cutoff.  It  appears  that  at  least  84%  of  the 
program  offices  do  not  complete  their  analyses  until  at  least  four  1 4 )  weeks 
after  the  cutoff;  and  at  least  63%  do  not  complete  their  analyses  until  at 
least  five  (5)  weeks  after  the  cutoff.  This  result  is  the  combination  of  the 
length  of  time  to  receive  the  reports  and  the  length  of  time  to  complete  the 
reviews  and  analyses  of  the  reports. 

Government  business  managers  were  asked  to  approximate  the  monthly  number  of 
total  program  office  man-days  spent  on  collecting,  reviewing,  analyzing  and 
summarizing  information  contained  in  CPRs  (GuM-24).  The  responses  are  shown 
in  Table  6.19,  differentiated  by  contract  value. 
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TABU  6.19 


PROGRAM  OFFICt  MAN-DAYS  USED  FOR  MONTHLY  CPR  ANALYSIS 


Monthly  Manpower 
Used  for  CPR 
Analysis 

Contract  Values 

Less  Than 
$40  Million 

At  least  540  Million 
but  less  than 
$160  Million 

5160  Mi  1 1  ion 
or  more 

Overal 1 

Man-days 

% 

% 

% 

% 

Less  Than  1 

5 

5 

3 

4 

1  -  4 

67 

44 

28 

41 

5-9 

19 

29 

25 

25 

10  -  14 

9 

13 

27 

19 

15  or  More 

0 

9 

17 

11 

TOTAL 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  government  business 
managers  who  indicated,  in  number  of  man-days,  the  manpower  used  for  CPR 
analysis  of  their  contract. 


It  is  apparent  from  the  data  that  the  number  of  man-days  used  for  CPR  reviews 
and  analyses  varies  substantially.  The  pattern  of  the  data  also  suggests  that 
the  number  of  man-days  used  for  CPR  analysis  increases  in  relation  to 
increasing  contract  value.  For  example,  with  contracts  valued  at  less  than 
$40  million,  72%  of  the  respondents  indicated  that  four  (4)  man-days  or  less 
were  used  each  month.  For  contracts  valued  at  approximately  540-160  million, 
only  49%  of  the  respondents  indicated  that  four  (4)  man-days  or  less  were  used 
each  month  for  CPR  analyses;  while  51%  indicated  that  five  (5)  man-days  or 
more  were  used.  For  contracts  valued  at  $160  million  or  more,  69%  indicated 
that  five  (5)  inan-days  or  more  were  used  each  month  for  CPR  analyses. 

Government  program  managers  were  asked  how  often  they  personally  reviewed  the 
cost/schedule  status  of  their  contracts  based  on  CPRs  or  data  derived  from 
CPRs  (GPM-8).  They  were  also  asked  how  often  they  used  this  information  in 
either  briefings  or  written  reports  to  higher  headquarters  (GPM-9).  The 
responses  are  shown  in  Table  6.20. 
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TABlfc.  fa. 20 


PROGRAM  MANhGLR  USt  OF  CPR  DATA 


Government  Program  Managers 

Frequency 

Personal ly 
Revi ew 

Used  for 

Hiqher  Headquarters 

% 

% 

More  Often  Than  Monthly 

17 

8 

Monthly 

68 

3fa 

Bimonthly 

fa 

9 

Quarterly 

5 

32 

Less  Often  Than  Quarterly 

3 

10 

Never 

1 

5 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  government 
program  managers  who  personally  review  CPR-derived  data  and  who  use 
CPR-derived  data  for  higher  headquarters,  for  the  frequencies  shown. 


Most  government  program  managers  (85%)  indicated  that  they  personally  reviewed 
cost/schedule  data  derived  from  CPRs  on  a  monthly  or  more  frequent  basis.  The 
same  percentage  (,85%)  indicated  that  they  used  information  derived  from  CPRs 
for  briefings  or  written  reports  to  higiier  headquarters  on  a  quarterly  or  more 
frequent  basis. 

bovernment  program  managers  also  were  asked  if  they  regularly  used  other 
reports  or  other  means  in  lieu  of  CPR-derived  data  in  order  to  determine 
contract  cost/schedule  status  (GPM-6).  The  majority  157%)  indicated  that  they 
did  not  use  other  reports  or  other  means.  However,  a  substantial  percentage 
(43%)  indicated  that  they  did  use  other  reports  or  means. 

The  response  to  this  question  was  a  key  discriminator  for  responses  to 
numerous  government  program  manager  questions  relating  to  perceptions  and 
assessments  towards  C/SCSC  and  CPRs.  For  example,  respondents  who  indicated 
they  used  other  reports  or  other  means  in  lieu  of  CPR-derived  data  in  order  to 
determine  contract  cost/schedule  status  had  ratings  and  favorable  perceptions 
towards  C/SCSC  and  CPRs  that  were  always  lower  than  for  those  who  did  not  use 
other  reports  or  means.  This  difference  was  frequently  significant  at  the  95% 
confidence  level. 
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Contractor  business  managers  were  askea  how  effectively  (very  effectively, 
somewhat  effectively,  or  not  at  all  effectively)  the  L)UU  program  office  to 
which  they  submitted  their  CPR  used  the  data  (CBM-lB).  For  those  who 
indicated  an  opinion,  88%  responded  that  the  Uuu  program  office  used  the  data 
effectively,  while  only  12%  responded  "not  at  all  effectively". 

6.3.6  Contractor  Use  of  Interna)  Cost/Schedule  Reports 

Contractor  program  managers  were  asked  how  often  they  personally  reviewed 
reports  of  any  type  either  generated  directly  by  their  C/SCSC-accepted  system 
or  derived  from  their  system  (CPM-8).  Contractor  business  managers  were  asked 
how  often  their  company  program  manager  personally  reviewed  these  reports 
C CBM- 23 ) .  The  results  are  shown  in  Table  6.21. 


TABLE  6.21 

PROGRAM  MANAGER  REVIEW  OF  C/SCSC-RELATED  REPORTS 


Frequency 

Contractor  Program  Manager 
Personally  Reviews  Reports 
According  to  According  to 
Prog.  Mqr.  Bus.  Mqr. 

% 

% 

Dai  ly 

4 

3 

Weekly 

46 

40 

Biweekly 

10 

12 

Monthly 

39 

45 

Quarterly 

1 

0 

TOTAL 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of 
contractor  program  managers  who  personally  review 
reports  generated  or  derived  from  their  C/SCSC-accepted 
systems,  for  the  frequencies  shown. 
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Most  contractor  program  managers  (60%)  indicated  that  they  personally  review 
C/SCSC-related  reports  of  any  type  on  a  biweekly  or  more  frequent  basis. 

Nearly  10U%  indicated  that  they  personally  review  such  reports  on  a  monthly  or 
more  frequent  basis. 

Contractor  program  managers  were  asked  if  they  regularly  used  other  reports  or 
other  means  in  lieu  of  using  C/SCSC-dr iven  internal  cost  and  schedule  reports 
in  order  to  determine  the  cost  and  schedule  status  of  their  contracts 
(CPM-6).  The  majority  (54%)  indicated  that  they  dia  use  other  reports  or 
other  means. 

Like  the  response  by  government  program  managers  to  a  very  similar  question 
(whether  they  regularly  used  other  reports  or  means  in  lieu  of  CPR-derived 
data),  the  contractor  program  manager  response  to  this  question  was  a 
discriminator  for  their  responses  to  numerous  other  questions  relating  to 
perceptions  and  assessments  towards  C/SCSC  and  CPRs.  For  contractor  program 
managers  who  indicated  they  used  other  reports  or  means  to  determine  the  cost 
and  schedule  status  of  their  contracts,  ratings  and  favorable  perceptions 
towards  C/SCSC  and  CPRs  were  normally  lower  than  for  those  who  did  not  use 
other  reports  or  means.  This  difference  was  frequently  significant  at  the  95% 
confidence  level. 


Contractor  business  managers  were  asked  if  their  company  had  an  in-house 
recurring  training  program  for  managers  on  how  to  use  internal  reports  from 
their  C/SCSC-accepted  system  (C8M-36).  Most  of  the  respondents  (71%) 
indicated  that  their  company  did  have  such  a  training  program. 


Summan 


Based  on  the  respondent  data,  we  conclude  that: 


The  lowest  CWBS  level  regularly  reported  on  CPRs  for  most  contracts 
was  CWBS  level  3  or  higher.  However,  many  CPRs  had  lower  level 
reporting.  Factors  which  related  to  differences  in  the  depth  of 
CWBS  reporting  levels  appeared  to  include  the  type  of  contract  worK 
(i.e.,  development  or  production),  the  contract  value,  and  whether 
the  contract  was  cost-plus  or  fixed-price  incentive.  Most 
government  and  contractor  business  managers  felt  that  their  CWBS 
reporting  levels  were  "just  about  right". 


While  most  contractor  business  managers  did  not  believe  that 
excessive  CPR  data  was  being  requested,  it  was  apparent  that  they 
felt  the  Problem  Analysis  format,  more  than  other  formats,  contained 
excessive  data.  Most  government  business  managers  indicated  that 
the  number  of  CPR  pages  could  not  be  reduced  without  hampering  their 
ability  to  manage  the  contract. 


For  internally  generated  contractor  cost/schedule  reports,  most 
contractor  business  managers  felt  that  some  reductions  could  ue  made 
in  the  number  of  pages  being  generated,  without  hampering  their 
ability  to  manage  their  contracts.  Some  contractor  business 
managers  felt  that  substantial  reductions  could  be  made.  In 
addition,  the  data  appeared  to  suggest  that  contractor  business 


manager  perceptions  of  government  needs  for  CPR  data  were  related  to 
whether  the  contractor  business  managers  believed  that  they, 
themselves,  were  generating  internal  reports  in  excess  of  their  own 
needs . 

Most  CPRs  are  forwarded  to  the  government  within  four  (4)  weeks 
after  the  close  of  the  contractor's  accounting  period.  Most  CPRs 
are  received  by  government  program  offices  within  five  (5)  weeks 
after  the  close  of  the  contractor's  accounting  period.  While  most 
government  business  managers  appear  to  be  satisifed  with  CPR 
timeliness,  most  government  program  managers  are  not  satisfied.  It 
appears  from  the  data  that  CPRs  should  be  received  within  four  (4) 
weeks  for  government  program  managers  to  feel  that  the  reports  are 
sufficiently  timely. 

For  contractor  internal  C/SCSC-related  reports,  most  contractors 
provided  the  reports  to  their  cost  account  managers  within  two  (2) 
weeks  after  the  reporting  period  cutoff  date.  More  than  one-fourth 
of  the  contractors  who  had  undergone  SARs  indicated  that  these 
reports  were  provided  within  the  first  week.  Most  contractor 
program  and  business  managers  agreed  that  their  internal  cost  and 
schedule  reports  were  sufficiently  timely  to  be  useful  to  them.  The 
data  also  suggested  that  most  business  managers  considered  their 
internal  cost/schedule  reports  sufficiently  timely  to  be  useful  when 
the  reports  were  provided  to  their  cost  account  managers  within 
three  (3)  weeks  after  the  close  of  the  reporting  period. 

Most  government  business  managers  indicated  that  their  program 
offices  reviewed  the  CPR  each  month  in  its  entirety.  Most 
government  business  managers  also  indicated  that  their  CPR  analyses 
were  completed  within  two  (2)  weeks  after  receipt  of  the  CPR.  For 
government  business  managers  whose  program  offices  complete  their 
analyses  within  two  (2)  weeks,  rather  than  two  (2)  weeks  or  more,  a 
greater  percentage  apparently  rate  their  CPRs  as  more  helpful  in 
determining  contract  status.  Most  program  offices  do  not  complete 
their  CPR  reviews  until  after  the  following  report  period's  cutoff. 
This  result  is  the  combination  of  the  length  of  time  to  receive  the 
reports  and  the  length  of  time  to  complete  the  analyses. 

The  monthly  number  of  total  program  office  man-days  used  for  CPR 
analyses  appeared  to  vary  with  contract  value.  For  contracts  valued 
at  less  than  $40  million,  most  government  business  managers 
indicated  that  four  (4)  man-days  or  less  were  used.  For  contracts 
valued  at  approximately  $40-160  million,  approximately  one-half  of 
the  government  business  managers  indicated  four  (4)  man-days  or  less 
and  one-half  indicated  five  (5)  man-days  or  more.  For  contracts 
valued  at  $160  million  or  more,  most  government  business  managers 
indicated  five  (5)  man-days  or  more  were  used  each  month  for  CPR 
analyses . 

Most  government  program  managers  indicated  that  they  personally 
reviewed  cost/schedule  data  derived  from  CPRs  on  a  monthly  or  more 
frequent  basis.  Most  government  program  managers  also  indicated 
that  they  used  information  derived  from  CPRs  for  briefings  or 
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written  reports  to  higher  headquarters  on  a  quarterly  or  more 
frequent  basis.  A  majority  of  government  program  managers  indicated 
that  they  did  not  use  other  means  in  lieu  of  CPR -derived  data  in 
order  to  determine  contract  cost/scheaule  status. 

Most  contractor  business  managers  felt  that  the  ODD  program  office 
to  which  they  submitted  their  CPR  used  the  data  effectively. 

Most  contractor  program  managers  review  C/SCSC-related  reports  of 
any  type  on  a  biweekly  or  more  frequent  basis,  nearly  10l)%  review 
such  reports  on  a  monthly  or  more  frequent  basis.  However,  a 
majority  of  contractor  program  managers  indicated  that  they 
regularly  used  other  means  in  lieu  of  using  their  C/SCSC-dr iven 
internal  reports  to  determine  the  cost  and  schedule  status  of  their 
contracts.  Most  contractor  business  managers  indicated  that  their 
companies  had  in-house  recurring  training  programs  for  their 
managers  on  how  to  use  internal  reports  from  their  C/SCSC-accepted 
systems. 
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7.0  C/SCSC  SURVEILLANCE 

7.1  Background 


C/SCbC  surveillance  is  frequently  cited  as  a  concern.  It  is  an  activity  that 
is  sometimes  thought  of  as  "in-between"  the  government  program  office  and  the 
contractor . 

C/SCSC  surveillance  is  the  reponsib i 1 i ty  of  the  cognizant  contract 
administration  office,  with  assistance  by  the  resident  DCAA  office. 
Surveillance  people  must  interface  with  both  the  government  and  contractor 
program  office  people.  To  be  effective,  surveillance  people  should  have  the 
confidence  and  support  of  the  government  program  office  and  the  professional 
respect  and  cooperation  of  the  contractor. 

Surveillance  is  normally  classifed  in  two  (2)  phases.  Prior  to  completion  of 
the  demonstration  process  an;  C/SCSC-acceptance  of  the  contractor's  system, 
the  in-plant  surveillance  effort  is  primarily  associated  with  assisting  in  the 
C/SCSC  review  process  and  with  the  resident  monitoring  of  contractor 
progress.  For  the  second  phase,  which  is  subsequent  to  completion  of  the 
demonstration  process  and  C/SCSC-acceptance,  in-plant  government  surveillance 
should  assume  the  more  predominant  role  through  recurring  evaluations  of  the 
contractor's  system.  The  purpose  is  to  assure  that  the  system  continues  to 
operate  properly. 

7.2  Overview  of  Related  Questions 

In  our  examination  of  C/SCSC  surveillance,  we  included  questions  which 
addressed  perceptions  and  assessments  in  the  three  (3)  areas  listed  below. 
Respondents  were  government  program  and  business  managers  and  contractor 
business  managers,  depending  on  the  specific  question. 

Government  program  office  perceptions  regarding  in-plant  C/3CSC 
surveillance  effectiveness. 

Assessments  reyarding  frequency  of  C/bCSC  surveillance  monitor 
contact  with  the  company. 

Contractor  perceptions  regaruiny  the  level  of  priority  and  ueyree  of 
expertise  for  C/SCbC,  within  the  government  plant  representative 
offices . 

7.3  Results 

7.3.1  C/bCbC  burveillance  Effectiveness 

Government  program  and  business  managers  were  asked  if  they  agreed  that 
non-program  office  plant  representat i ves  such  as  DCaB,  UCAm,  NAVPRUs,  etc.,  oo 
a  reasonably  good  job  of  C/SCbC  surveillance  (GPM-17  and  GBM-32).  The  results 
are  shown  in  Table  7.1. 
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TABLE  7.1 


C/SCSC  SURVEILLANCE  EFFECTIVENESS 


Non-Program  Office  Plant 
Representatives  Oo  a 
Reasonably  Good  Job  of 

C/SCSC  Surveillance 

Government 

Proq.  Mqr. 

Business  Manaqer 

Overal  T 
% 

Demonstration 

% 

SAR 

% 

Overal l 
% 

Agree 

68 

59 

79 

73 

Disagree 

26 

33 

18 

22 

Neither  Agree 

Nor  Disagree 

6 

8 

3 

5 

total 

100 

100 

100 

100 

The  numerical  values  in  this  table  show  the  percent  of  respondents  who  agreed 
or  disagreed  regarding  whether  non-program  office  plant  representatives  did  a 
reasonably  good  job  at  C/SCSC  surveillance.  In  addition,  business  manager 
responses  are  differentiated  by  whether  their  contract  had  undergone  a 
demonstration  or  SAR. 


Most  government  program  and  business  managers  agreed  that  non-program  office 
plant  representatives  do  a  reasonably  good  job  at  C/SCSC  surveillance  (68%  for 
program  managers  and  73%  for  business  managers).  The  responses  from 
government  business  managers  also  appear  to  suggest  stronger  agreement  that 
plant  representatives  do  a  reasonably  good  job  at  surveillance  when  their 
contracts  have  undergone  SARs,  rather  than  demonstrations.  This  would 
normally  occur  when  contractors  have  previously  accepted  systems,  i.e.,  during 
the  second  phase  of  surveillance  (when  the  in-plant  activity  normally  assumes 
a  more  predominant  role). 

7.3.2  Frequency  of  Contact  Between  the  Government  C/SCSC  Surveillance  Monitor 
and  the  Company 

Contractor  business  managers  were  asked  how  frequently  the  government  C/SCSC 
surveillance  monitor  has  contact  with  the  company  (CBM-35).  The  results  are 
shown  in  Table  7.2. 
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TABLE  7.2 


GOVERNMENT  C/SCSC  SURVEILLANCE  MONITOR  INTERFACE  WITH  COMPANY 


C/SCSC  Surveillance  Monitor's 
Frequency  of  Contact 

With  the  Company 

Contractor  Business 
Demonstration 
% 

Manager 
SAk  ~ 

1 r 

Daily 

15 

22 

Week  1 y 

26 

28 

Monthly 

48 

39 

Less  Often  Than  Monthly 

11 

11 

TOTAL 

100 

100 

The  numerical  values  in  the  table  show  the  percent  of  contractor 
business  managers  who  indicated  the  C/SCSC  surveillance  monitor's 
frequency  of  contact  with  the  company,  as  shown.  The  base  does  not 
include  the  respondents  who  answered  "Don't  Know". 


Most  of  the  contractor  business  managers  (89%)  indicated  that  the  government 
C/SCSC  surveillance  monitor  had  monthly  or  more  frequent  contact  with  the 
company.  For  contracts  which  had  undergone  SARs  (contractors  with  previously 
accepted  systems  and  in  the  second  phase  of  surveillance),  50%  of  the 
contractor  business  managers  indicated  that  the  government  C/SCSC  surveillance 
monitor  had  weekly  or  more  frequent  contact  with  the  company.  For  contracts 
which  had  undergone  demonstrations,  only  41%  indicated  this  same  frequency  of 
contact.  These  results  suggest  that  for  contractors  with  previously  accepted 
systems  and  in  the  second  phase  of  surveillance,  the  frequency  of  contact  by 
the  government  C/SCSC  surveillance  monitor  is  greater  than  for  contractors 
that  have  just  undergone  demonstrations.  This  appears  consistent  with  the 
anticipated  roles  of  surveillance  people  --  being  more  predominant  after 
system  acceptance  or  during  the  second  phase  of  surveillance. 

7.3.3  Government  In-Plant  C/SCSC  Priority  and  Expertise 

Contractor  business  managers  were  asked  if  they  agreed  that  in-plant 
government  representatives  have  a  variety  of  responsibilities,  with  C/SCSC 
usually  being  low  in  priority  (CBM-27).  They  also  were  asked  if  they  agreed 
that  in-plant  government  people  do  not  have  much,  if  any,  C/SCSC  expertise 
(CBM-28).  The  results  are  shown  in  Table  7.3. 
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TABLE  7.3 

GOVERNMENT  IN-PLANT  C/SCSC  PRIORITY  AND  EXPERTISE 


Statements  Regarding 
Government  In-Plant 

C/SCSC  Activities 

Contractor  Business  Manager 

Demonstration 

% 

SAR 

% 

Uveral  l 

— r 

Usually  Low  in  Priority 

58 

41 

47 

People  Do  Not  Have  Much  Expertise 

64 

39 

45 

The  numerical  values  in  this  table  show  the  percent  of  contractor 
business  managers  who  agreed  with  the  statement  as  shown. 


For  contracts  which  had  undergone  SARs  (systems  previously  accepted  as 
C/SCSC-compliant  and  in  the  second  phase  of  surveillance),  less  than  the 
majority  of  contractor  business  managers  (41%)  agreed  that  in-plant  government 
representatives  had  a  variety  of  responsibilities  with  C/SCSC  usually  being 
low  in  priority;  and  less  than  the  majority  (39%)  agreed  that  in-plant 
government  people  did  not  have  much  expertise  in  C/SCSC.  However,  for 
contracts  which  had  just  undergone  demonstrations,  the  majority  agreed  that 
C/SCSC  was  usually  low  in  priority  (58%)  and  that  the  people  did  not  have  much 
expertise  (64%).  The  differences  in  both  cases  show  more  favorable 
perceptions  of  the  in-plant  government  C/SCSC  activity  for  contracts  which  had 
undergone  SARs  (during  the  second  phase  of  surveillance)  than  for  contracts 
which  had  undergone  demonstrations.  These  differences  are  statistically 
significant  at  the  95%  confidence  level. 

The  less  favorable  perceptions  of  the  in-plant  C/SCSC  surveillance  activity 
during  the  earlier  phase  of  surveillance  may  be  due  to  such  factors  as  an 
apparently  more  subordinate  role  being  played  during  this  phase  and  a  real 
lack  of  expertise  which  becomes  remedied  as  the  review  process  progresses. 

7.4  Summary 

Based  on  the  respondent  data,  we  conclude  that: 

Most  government  program  and  business  managers  agree  that  government 
plant  representatives  do  a  reasonably  good  job  at  C/SCSC 
surveillance.  Agreement  appears  to  be  stronger  when  contractors  are 
more  likely  to  have  previously  accepted  systems  and  surveillance  has 
a  more  predominant  role. 


Ill  -  75 


i 


Most  C/SCSC  surveillance  monitors  have  at  least  monthly  contact  with 
company  representatives.  For  contractors  with  systems  previously 
accepted  as  C/SCSC-compliant  or  in  the  second  phase  of  surveillance, 
it  appears  that  approximately  one-half  of  the  surveillance  monitors 
have  at  least  weekly  contact,  while  about  20%  to  25%  of  surveillance 
monitors  have  daily  contact. 

For  contractors  with  previously  accepted  systems  or  in  the  second 
phase  of  surveillance,  less  than  the  majority  of  contractor  business 
managers  felt  that  in-plant  government  C/SCSC  surveillance  was  low 
in  priority  and  that  in-plant  government  people  were  lacking  in 
C/SCSC  expertise.  For  contractors  that  had  just  undergone  the 
demonstration  process,  the  contractor  business  managers  felt 
differently.  They  felt  that  C/SCSC  was  low  in  priority  and  that  the 
in-plant  government  people  were  lacking  in  C/SCSC  expertise.  This 
would  appear  to  reflect  such  factors  as  an  apparently  more 
subordinate  role  of  surveillance  during  the  earlier  phase  and  a  real 
lack  of  expertise  which  becomes  remedied  as  the  review  process 
progresses. 
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SECTION  IV 


CONCLUSIONS 


INTRODUCTION 


These  Phase  I  survey  conclusions  follow  directly  from  the  results  and 
summaries  contained  In  Section  III  of  this  report.  Although  recommendations 
are  not  included  within  this  report,  recommendations  have  been  developed  for 
the  Phase  II  portion  of  the  survey.  The  recommendations  for  Phase  II  are 
focused  towards  respondent  concerns  identified  in  these  conclusions. 
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1.0  COST  AND  BENEFIT 


C/SCSC-accepted  systems  are  considered  to  be  effective  in  assisting  to  control 
cost  and  schedule  performance. 

C/SCSC-accepted  systems  are  considered  to  be  effective  by  a  large  majority  of 
the  respondents  (71%  for  contractor  program  managers  and  80%  for  government 
business  managers).  A  large  majority  of  contractor  program  managers  (74%)  and 
government  program  managers  (77%)  also  perceive  that  a  major  benefit  of  C/SCSC 
is  more  thorough  contractor  planning  than  otherwise  would  be  accomplished. 

CPRs  and  related  internal  contractor  reports  are  considered  to  be  accurate. 
CPRs  are  considered  most  useful  for  determining  contract  cost  status. 

Over  90%  of  the  contractor  program  and  business  managers  considered  their  CPRs 
to  be  accurate.  Most  government  business  managers  (80%)  also  agreed  that 
their  contractor's  internal  data  provides  valid  information.  Approximately 
75-80%  of  the  respondents  considered  CPRs  and  related  internal  reports  as 
good-to-excel lent  in  helping  to  determine  contract  cost  status.  A  majority 
(50%  or  more)  of  government  business  managers  and  contractor  program  managers 
also  considered  these  reports  as  helpful  for  estimates-at-completion, 
determining  cost  impacts  of  known  problems,  and  tracing  problems  to  their 
sources.  CPRs  are  not  as  helpful,  however,  in  these  areas  to  government 
program  managers. 

C/SCSC  benefits  are  considered  to  outweigh  the  costs.  Nevertheless,  most 
contractor  program  managers  see  a  need  for  minor  modifications  to  their 
systems.  Many  government  program  managers  agree. 

All  four  categories  of  managers  agreed  (ranging  from  53%  to  62%)  that  C/SCSC 
benefits  to  themselves  outweighed  its  associated  cost.  However,  the  strength 
of  this  belief  was  less  than  that  relating  solely  to  C/SCSC  effectiveness. 

When  program  managers  are  given  the  choice  of  whether  they  would  require  or 
use  current  C/SCSC-accepted  systems  as  is,  or  with  some  modification,  more 
government  program  managers  (39%)  than  contractor  program  managers  (21%)  would 
require  or  use  these  systems  as  is.  Nevertheless,  a  plurality  of  government 
program  managers  (41%)  and  a  majority  of  contractor  program  managers  (57%) 
perceive  the  need  for  at  least  some  minor  system  modifications.  The  great 
majority  of  both  government  (80%)  and  contractor  program  managers  (78%)  do  not 
perceive  the  need  for  major  modifications. 
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Manpower  cost  for  operating  C/SCSC-compl i ant  systems  (not  the  cost  of  C/SCSC) 
is  approximated  as  10  man-years  per  year  or  less  for  nearly  half  of 
contracts.  Manpower  cost  appears  to  vary  directly  with  contract  value. 

Eighteen  percent  ( 1 8% )  of  the  contractor  business  managers  approximated  the 
cost  of  operating  their  C/SCSC-accepted  systems  (not  the  cost  of  C/SCSC)  as  5 
man-years  per  year  or  less.  Nearly  one-half  (46%)  approximated  this  cost  as 
10  man-years  per  year  or  less;  while  two-thirds  (67%)  and  three-fourths  (76%) 
approximated  the  cost  as  15  and  20  man-years  per  year  or  less,  respectively. 
Patterns  in  the  data  suggest  that  perceptions  of  whether  benefits  outweigh 
costs  appear  to  be  inversely  related  to  the  cost  of  operating  C/SCSC-accepted 
systems.  Additionally  the  data  suggests  that  these  costs  vary  directly  with 
the  dollar  value  of  the  applicable  contract. 
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2.0  CRITERIA  CONCEPT  AND  APPROACH 


Criteria  concept  is  considered  to  be  valid. 

The  criteria  concept  is  considered  valid  by  a  large  majority  (ranging  from  77% 
to  88%)  of  the  respondents  (government  program  managers;  and  contractor 
program  and  business  managers). 

Currently  delineated  criteria  are  considered  to  be  appropriate. 

Most  managers  in  all  four  (4)  categories  (government  program  and  business 
managers;  and  contractor  program  and  business  managers)  considered  the 
currently  delineated  criteria  appropriate  as  the  basis  for  evaluating 
contractor  systems.  While  most  contractor  program  (63%)  and  business  managers 
(69%)  considered  the  criteria  appropriate,  the  percent  was  not  as  high  as  for 
their  counterpart  government  program  (71%)  and  business  managers  (77%).  In 
addition,  for  contractor  managers,  there  appeared  to  be  less  agreement  that 
the  currently  delineated  criteria  are  appropriate  than  that  the  criteria 
concept  is  valid. 

Approach  of  ensuring  system  adequacy  is  considered  to  be  as  good  or  better 
than  other  approaches. 

A  large  majority  (ranging  from  73%  to  77%)  of  the  respondents  (government 
program  managers;  and  contractor  program  and  business  managers)  agreed  that 
the  approach  of  ensuring  that  an  adequate  basis  exists  for  keeping  the  program 
office  informed  is  as  good  or  better  than  other  approaches  to  achieving  the 
same  objective. 

POP  practices  in  ensuring  that  contractor  systems  are  adequate  are  considered 
to  be  effective. 

A  large  majority  in  all  four  (4)  categories  (government  program  and  business 
managers;  and  contractor  program  and  business  managers)  considered  00D 
practices  effective  in  ensuring  that  an  adequate  basis  exists  in  a 
contractor's  system  for  reporting  cost/schedule  status.  While  a  large 
majority  of  contractor  program  (67%)  and  business  managers  (75%)  considered 
DOD  practices  effective,  the  percent  was  not  as  high  as  for  the  government 
program  (74%)  and  business  managers  (82%).  On  a  general  level,  it  does  not 
appear  that  000  practices  are  overly  specified.  Conclusions  regarding 
specific  000  practices  as  they  relate  to  the  C/SCSC  review  process  and  depth 
of  reporting  are  discussed  in  paragraphs  3.0,  4.0  and  6.0  of  this  report 
section. 

Perceptions  of  criteria  concept  and  DOD  practices  are  more  favorable  for 
contractors  who  had  undergone  Subsequent  Application  Reviews  than  for 
contractors  who  had  undergone  Demonstrations. 

For  the  validity  of  the  criteria  concept  and  the  effectiveness  of  DOD 
practices,  contractor  business  managers  have  signif icantly  more  favorable 
perceptions  after  undergoing  the  Subsequent  Application  Review  process  than 
after  undergoing  the  demonstration  process. 
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3.0  C/SCSC  REVIEW  PROCESS 


Several  government  organizations  participate  in  the  C/SCSC  review  process. 

000  review  teams  are  normally  organized  to  include  participation  of  the 
program  office,  government  plant  representatives,  resident  DCAA 
representatives,  service  focal  point  office  and  commodity  command. 

Participation  of  each  of  these  organizations  is  considered  to  be  necessary. 

Most  government  business  managers  feel  that  review  team  participation  is 
necessary  for  each  of  the  organizations  which  currently  provide  team  members. 
There  was  nearly  unanimous  agreement  regarding  the  necessity  for  program 
office  (99%),  contract  administration  office  (97%)  and  DCAA  participation 
(94%). 

Most  Demonstrations  are  completed  within  three  weeks.  Most  Subsequent 
Application  Reviews  ($ARs)  are  completed  within  one  week. 

For  Demonstrations,  approximately  one-half  of  the  reviews  are  completed  within 
two  weeks  and  more  than  four-fifths  are  completed  within  three  weeks.  Most 
SARs  (approximately  70%)  are  completed  within  one  week. 

The  predominant  number  of  review  team  members  is  8-11  persons.  Team  size 
appears  to  be  larger  for  Demonstrations  than  for  SARs. 

The  predominant  number  of  review  team  members  is  8-11  persons  (approximately 
40-45%  of  teams).  For  Demonstration  Reviews,  approximately  60%  of  the 
respondents  indicated  the  review  team  size  was  11  persons  or  less.  For  SARs, 
approximately  three-fourths  of  the  government  business  managers  and  two-thirds 
of  the  contractor  business  managers  indicated  the  review  team  size  was  11 
persons  or  less. 

There  appears  to  be  a  greater  number  of  contractor  discrepancies  identified 
during  Demonstrations  than  identified  during  SARs. 

For  Demonstrations,  10  discrepancies  or  less  (requiring  contractor  corrective 
e-tions)  are  identified  during  approximately  one-half  of  the  reviews.  For 
SARs,  10  discrepancies  or  less  are  identified  during  approximately  two-thirds 
of  the  reviews. 

Most  contractors  consider  POD  C/SCSC  review  teams  to  be  doing  a  good-to- 
excellent  job  when  evaluating  contractor  systems. 

The  majority  of  contractor  business  managers  feel  that  DOD  review  teams  do  a 
good-to-excel lent  job  when  evaluating  their  company's  C/SCSC  application.  The 
largest  majorities  (approximately  80%)  were  for  the  three  characteristics  of 
overall  effectiveness,  leadership  and  thoroughness.  The  smallest  majorities 
(approximately  60%)  were  for  the  two  characteristics  of  flexibility  and 
technical  qualifications. 
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Contractor  manpower  costs  solely  attributable  to  the  C/SCSC  review  process 
appear  significantly  higher  in  support  of  the  demonstration  process  than  fn 
support  of  the  SAR  process. 

For  costs  solely  attributable  to  the  conduct  of  C/SCSC  reviews,  the  contractor 
business  manager  responses  appear  to  show  that  more  support  is  required  for 
the  demonstration  process  than  for  SARs.  For  example,  100  man-days  or  less 
was  estimated  for  25%  of  the  demonstration  processes  and  for  53%  of  the  SAR 
processes. 

For  contractors  who  had  undergone  demonstrations,  the  benefits  of  the  review 
process  are  considered  to  outweigh  the  costs.  For  contractors  who  had 
undergone  SARs,  the  costs  of  the  review  process  are  considered  to  outweigh  the 
benefits. 

The  majority  of  contractor  business  managers  (52%;  39%  indicated  the  opposite, 
and  9%  indicated  "No  opinion")  indicated  that  the  benefits  to  their  company  of 
the  demonstration  process  outweigh  its  costs  (costs  solely  attributable  to  the 
reviews).  However,  for  SARs,  the  majority  of  contractor  business  managers 
(55%;  30%  indicated  the  opposite,  and  15%  indicated  "No  opinion")  indicated 
that  the  benefits  did  not  outweigh  the  costs. 

Government  business  managers  consider  the  C/SCSC  review  process  to  be 
necessary  for  the  proper  functioning  of  the  contractor's  cost  and  schedule 
control  system. 

Government  business  managers  were  nearly  unanimous  (98%)  in  indicating  that 
the  C/SCSC  review  process  was  necessary.  This  was  true  for  both  the 
demonstration  process  and  SAR  process.  Agreement  regarding  the  need  for  the 
demonstration  process  appeared  to  be  slightly  stronger  than  for  the  SAR 
process. 
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4.0  CONSISTENCY  OF  C/SCSC  INTERPRETATION 


Contractor  business  managers  consider  C/SCSC  interpretations  by  POP  review 
teams  to  be  consistent. 

Most  contractor  business  managers  feel  that  their  review  team's  C/SCSC 
interpretations  were  reasonably  consistent  with  other  DOD  review  teams' 
interpretations.  The  pattern  of  the  data  appears  to  suggest  that  there  is 
higher  agreement  regarding  team  consistency  for  those  contractors  who  had 
undergone  SARs  ( 71%)  than  for  those  who  had  undergone  Demonstration  Reviews 
[58%).  For  those  respondents  who  answered  "yes"  or  "no"  (rather  than  "no 
opinion")  regarding  whether  review  teams  were  reasonably  consistent,  79%  of 
those  who  went  through  demonstrations  and  90%  of  those  who  went  through  SARs 
answered  "yes". 
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5.0  TIME  ALLOWANCES  FOR  IMPLEMENTING  C/SCSC 


Most  C/SCSC-related  performance  measurement  baselines  are  established  by 
contractors  within  six  (6)  months  of  contract  award.  This  accomplishment 
appears  to  take  longer  for  contracts  undergoing  demonstrations  than  for  those 
undergoing  SARs. 

The  C/SCSC-related  performance  measurement  baselines  for  approximately 
two-thirds  of  the  contracts  were  established  within  six  (6)  months  of  contract 
award.  According  to  contractor  business  managers,  it  appears  to  take  longer 
to  establish  the  baselines  for  contracts  undergoing  the  demonstration  process 
than  for  contracts  undergoing  the  SAR  process.  For  example,  contractor 
respondents  indicated  that  for  demonstrations,  52%  of  the  baselines  were 
established  within  six  (6)  months;  while  for  SARs,  76%  were  established  within 
six  (6)  months.  Since  contractors  undergoing  SARs  should  have  a  previously 
proven  capability  for  developing  a  baseline,  this  should  be  expected. 

C/SCSC-related  performance  measurement  baselines  appear  to  be  established 
earlier  when  contracts  are  definitized  at  contract  award.  For  undef initizeo 
contracts,  nearly  half  have  their  baselines  established  prior  to 
def initization. 

According  to  contractor  business  managers,  for  contracts  that  are  not 
definitized  at  contract  award,  it  appears  to  take  longer  to  establish  the 
performance  measurement  baseline  than  for  those  contracts  that  were 
definitized  at  contract  award.  For  example,  contractor  respondents  indicated 
that  for  contracts  definitized  at  contract  award,  82%  of  the  baselines  were 
established  within  six  (6)  months;  while  for  undef initized  contracts,  only  45% 
were  established  within  six  (6)  months.  It  also  appears  that  somewhat  less 
than  half  (contractor  respondents  indicated  46%)  of  the  undef initized 
contracts  have  their  baselines  established  prior  to  def initization. 

Government  business  managers  appear  satisfied  when  C/SCSC-related  performance 
measurement  baselines  are  established  within  six  (6)  months  of  contract 
award.  When  baselines  take  longer  to  establish,  most  of  the  respondents 
appear  to  feel  it  could  have  been  done  more  quickly. 

Most  government  business  managers  appear  to  be  satisfied  that  the  elapsed  time 
is  reasonable  when  performance  measurement  baselines  are  established  within 
six  (6)  months  of  contract  award  (78%  appear  satisfied  when  it  takes  less  than 
3-months;  and  73%  appear  satisfied  when  it  takes  at  least  3-months  but  less 
than  6-months).  Most  government  business  managers  also  appear  to  feel  that 
when  baselines  take  longer  than  six  (6)  months,  it  could  have  been  done  more 
quickly  (57%  appear  to  feel  it  could  have  been  done  more  quickly  when  it  takes 
at  least  6-months  but  less  than  12-months;  and  69%  appear  to  feel  this  way 
when  it  takes  12-months  or  more). 


A  negligible  number  of  C/SCSC  reviews  fall  within  ninety  (90)  calendar  days 
from  contract  award. 

A  negligible  number  of  C/SCSC  reviews  (none  for  demonstrations  and  2%  for 
SARs,  according  to  contractor  business  managers)  appear  to  fall  within  the 
ninety  (90)  calendar  day  time-frame  contained  within  the  DAR  C/SCSC  clause. 
Most  C/SCSC  visits  regarded  as  key  by  the  respondents  appeared  to  occur  within 
eighteen  (18)  months  of  contract  award  for  the  demonstration  process  and 
within  twelve  (12)  months  of  contract  award  for  the  SAR  process. 


6.0  COST /SCHEDULE  PERFORMANCE  REPORTING 


CWBS  level  3  is  the  lowest  level  regularly  reported  on  most  CPRs.  The  level 
f  of  reporting  appears  to  relate  to  factors  such  as  kind  of  work,  contract  value 

and  contract  type.  Most  respondents  felt  that  reporting  levels  were  "just 
about  right". 

The  lowest  CWBS  level  regularly  reported  on  CPRs  for  most  contracts  (69% 
according  to  government  business  managers  and  57%  according  to  contractor 
business  managers)  was  CWBS  level  3  or  higher.  However,  many  CPRs  had  lower 
level  reporting.  Factors  which  related  to  differences  in  the  depth  of  CWBS 
reporting  levels  appeared  to  include  the  type  of  contract  work  (i.e., 
development  or  production),  the  contract  value,  and  whether  the  contract  was 
cost-plus  or  fixed-price  incentive.  Most  government  (83%)  and  contractor 
business  managers  (77%)  felt  that  their  CWBS  reporting  levels  were  "just  about 
right". 

Most  contractor  business  managers  do  not  believe  that  the  five  (5)  CPR  formats 
I  contain  excessive  data.  Most  government  business  managers  indicated  that  the 

number  of  CPR  pages  could  not  be  reducecT 

While  most  contractor  business  managers  (ranging  from  82%  to  56%,  depending  on 
the  specific  CPR  format)  did  not  believe  that  excessive  CPR  data  was  being 
requested,  it  was  apparent  that  they  (44%)  felt  the  Problem  Analysis  format, 
more  than  other  formats,  contained  excessive  data.  Most  government  business 
managers  (79%)  indicated  that  the  number  of  CPR  pages  could  not  be  reduced 
without  hampering  their  ability  to  manage  the  contract. 

Most  contractor  business  managers  felt  that  some  reductions  could  be  made  in 
the  number  of  report  pacjes  generated  each  month  from  their  C/SCSC-accepted 
systems,  without  hampering  their  ability  to  manage. 

For  internally  generated  contractor  cost/schedule  reports,  most  contractor 
business  managers  (65%)  felt  that  some  reductions  could  be  made  in  the  number 
of  pages  being  generated,  without  hampering  their  ability  to  manage  their 
contracts.  Some  contractor  business  managers  (35%)  felt  that  substantial 
reductions  (21%  or  more  of  the  pages)  could  be  made.  In  addition,  the  data 
appeared  to  suggest  that  contractor  business  manager  perceptions  of  government 
needs  for  CPR  data  were  related  to  whether  the  contractor  business  managers 
believed  that  they,  themselves,  were  generating  internal  reports  in  excess  of 
their  own  needs. 

Most  CPRs  are  received  within  five  (5)  weeks  after  the  close  of  the  reporting 
period.  Although  most  government  business  managers  are  satisfied  with  this 
timeliness,  most  government  program  managers  are  not  satisifed. 

Most  CPRs  (77%)  are  forwarded  to  the  government  within  four  (4)  weeks  after 
the  close  of  the  contractor's  accounting  period.  Most  CPRs  (80%)  are  received 
by  government  program  offices  within  five  (5)  weeks  after  the  close  of  the 
contractor's  accounting  period.  While  most  government  business  managers  (61%) 
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appear  to  be  satisifed  with  CPR  timeliness,  most  government  program  managers 
(55%)  are  not  satisfied.  It  appears  from  the  data  that  CPRs  should  be 
received  within  four  (4)  weeks  for  government  program  managers  to  feel  that 
the  reports  are  sufficiently  timely. 

Most  internal  C/SCSC-related  reports  are  received  by  cost  account  managers 
within  two  (2)  weeks  after  the  close  of  the  reporting  period.  Most  contractor 
managers  consider  their  reports  to  be  sufficiently  timely^ 

For  contractor  internal  C/SCSC-related  reports,  most  contractors  (75%) 
provided  the  reports  to  their  cost  account  managers  within  two  (2)  weeks  after 
the  reporting  period  cutoff  date.  More  than  one-fourth  of  the  contractors  who 
had  undergone  SARs  indicated  that  these  reports  were  provided  within  the  first 
week.  Most  contractor  program  (72%)  and  business  managers  (74%)  agreed  that 
their  internal  cost  and  schedule  reports  were  sufficiently  timely  to  be  useful 
to  them.  The  data  also  suggested  that  most  business  managers  considered  their 
internal  cost/schedule  reports  sufficiently  timely  to  be  useful  when  the 
reports  were  provided  to  their  cost  account  managers  within  three  (3)  weeks 
after  the  close  of  the  reporting  period. 

Most  government  program  offices  review  CPRs  in  their  entirety.  Most 
government  CPR  analyses  are  completed  within  two  (2)  weeks  after  receipt  of 
the  CPR.  Most  CPR  analyses  are  not  completed  until  after  the  following  report 
period's  cutoff  date. 

Most  government  business  managers  (74%)  indicated  that  their  program  offices 
reviewed  the  CPR  each  month  in  its  entirety.  Most  government  business 
managers  (64%)  also  indicated  that  their  CPR  analyses  were  completed  within 
two  (2)  weeks  after  receipt  of  the  CPR.  For  government  business  managers 
whose  program  offices  complete  their  analyses  within  two  (2)  weeks,  rather 
than  two  (2)  weeks  or  more,  a  greater  percentage  apparently  rate  their  CPRs  as 
more  helpful  in  determining  contract  status  (e.g.,  93%  versus  65%, 
respectively  in  giving  a  rating  of  good-to-excel lent  for  determining  cost 
status).  Most  program  offices  (at  least  63%)  do  not  complete  their  CPR 
reviews  until  after  the  following  report  period's  cutoff.  This  result  is  the 
combination  of  the  length  of  time  to  receive  the  reports  and  the  length  of 
time  to  complete  the  analyses. 

Total  program  office  man-days  used  for  CPR  analysis  is  approximated  as  less 
than  10  man-days  per  month  for  70%  of  contracts.  Man-days  used  for  analysis 
ot  a  L'PK  appears  to  vary  with  contract  valued 

The  monthly  number  of  total  program  office  man-days  used  for  CPR  analyses 
appeared  to  vary  with  contract  value.  For  contracts  valued  at  less  than  $40 
million,  most  government  business  managers  (72%)  indicated  that  four  (4) 
man-days  or  less  were  used.  For  contracts  valued  at  approximately  $40-160 
million,  approximately  one-half  of  the  government  business  managers  indicated 
four  (4)  man-days  or  less  and  one-half  indicated  five  (5)  man-days  or  more. 

For  contracts  valued  at  $160  million  or  more,  most  government  business 
managers  (69%)  indicated  five  (5)  man-days  or  more  were  used  each  month  for 
CPR  analyses. 
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Most  government  program  managers  personally  review  CPR-related  data  at  least 
monthly.  Most  government  program  managers  use  CPR-related  information  for 
briefings  or  reports  to  higher  headquarters  at  least  quarterly. 

Most  government  program  managers  (85%)  indicated  that  they  personally  reviewed 
cost/schedule  data  derived  from  CPRs  on  a  monthly  or  more  frequent  basis. 

Most  government  program  managers  (85%)  also  indicated  that  they  used 
information  derived  from  CPRs  for  briefings  or  written  reports  to  higher 
headquarters  on  a  quarterly  or  more  frequent  basis.  A  majority  of  government 
program  managers  (57%)  indicated  that  they  did  not  use  other  means  in  lieu  of 
CPR-derived  data  in  order  to  determine  contract  cost/schedule  status. 

Most  contractors  feel  that  POD  program  offices  effectively  use  CPR  data. 

Most  contractor  business  managers  (88%)  felt  that  the  DOD  program  office  to 
which  they  submitted  their  CPR  used  the  data  effectively. 

Most  contractor  program  managers  personally  review  internal  C/SCSC-related 
reports  at  least  biweekly.  However t  most  contractor  program  managers  also 
regularly  use  other  means  to  determine  the  cost  and  schedule  status  of  their 
contracts.  Most  contractors  have  recurring  training  programs  on  how  to  use 
internal  reports. 

Most  contractor  program  managers  (60%)  review  C/SCSC-related  reports  of  any 
type  on  a  biweekly  or  more  frequent  basis.  Nearly  100%  review  such  reports  on 
a  monthly  or  more  frequent  basis.  However,  a  majority  of  contractor  program 
managers  (54%)  indicated  that  they  regularly  used  other  means  in  lieu  of  using 
their  C/SCSC-driven  internal  reports  to  determine  the  cost  and  schedule  status 
of  their  contracts.  Most  contractor  business  managers  (71%)  indicated  that 
their  companies  had  in-house  recurring  training  programs  for  their  managers  on 
how  to  use  internal  reports  from  their  C/SCSC-accepted  systems. 


7.0  C/SCSC  SURVEILLANCE 


Most  government  program  and  business  managers  feel  that  government  plant 
representatives  do  a  good  job  at  C/SCSC  surveillance.  This  view  appears 
stronger  when  the  contractor  is  likely  to  have  a  previously  accepted  system. 

Most  government  program  (68%)  and  business  managers  (73%)  agree  that 
government  plant  representatives  do  a  reasonably  good  job  at  C/SCSC 
surveillance.  Agreement  appears  to  be  stronger  when  contractors  are  more 
likely  to  have  previously  accepted  systems  and  surveillance  has  a  more 
predominant  role. 

Most  C/SCSC  surveillance  monitors  have  at  least  monthly  contact  with  company 
representatives.  For  contractors  with  previously  accepted  systems,  the 
frequency  of  contact  appears  greater. 

Most  C/SCSC  surveillance  monitors  (89%)  have  at  least  monthly  contact  with 
(  company  representatives.  For  contractors  with  systems  previously  accepted  as 

C/SCSC-compl iant  or  in  the  second  phase  of  surveillance,  it  appears  that 
approximately  one-half  of  the  surveillance  monitors  have  at  least  weekly  or 
more  frequent  contact,  while  about  20%  to  25%  of  surveillance  monitors  have 
daily  contact. 

Most  government  business  managers  do  not  feel  that  C/SCSC  surveillance  is  low 
in  priority  and  that  in-plant  people  are  lacking  in  C/SCSC  expertise,  when 
contractors  have  previously  accepted  systems.  However,  for  demonstrations, 
government  business  managers  feel  that  C/SCSC  surveillance  is  low  in  priority 
and  in-plant  surveillance  people  are  lacking  in  C/SCSC  expertise. 

For  contractors  with  previously  accepted  systems  or  in  the  second  phase  of 
surveillance,  less  than  the  majority  of  contractor  business  managers  felt  that 
in-plant  government  C/SCSC  surveillance  was  low  in  priority  (41%)  and  that 
in-plant  government  people  were  lacking  in  C/SCSC  expertise  (39%).  For 
contractors  that  had  just  undergone  the  demonstration  process,  the  contractor 
business  managers  felt  differently.  They  felt  that  C/SCSC  was  low  in  priority 
(58%)  and  that  the  in-plant  government  people  were  lacking  in  C/SCSC  expertise 
(64%).  This  would  appear  to  reflect  such  factors  as  an  apparently  more 
subordinate  role  of  surveillance  during  the  earlier  phase  and  a  real  lack  of 
expertise  which  becomes  remedied  as  the  review  process  progresses. 
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APPENDIX  1 


questionnaires  and  instructions 


This  appendix  contains  the  four  (4)  questionnaires  and  the  completion 
instructions.  The  introductory  letter  signed  by  Mr.  Paul  Thayer,  Deputy 
Secretary  of  Defense,  was  included  with  the  materials  sent  to  both  the 
government  and  contractor  program  managers. 


THE  DEPUTY  SECRETARY  OF  DEFENSE 


WASHINGTON.  D.C.  20301 


1  JUN  1983 


Dear  Program  Manager: 

Enclosed  are  questionnaires  that  relate  to  the  DoD  Cost/ 
Schedule  Control  Systems  Criteria  (C/SCSC).  They  are  being  sent 
to  all  Industry  and  DoD  program  managers  with  current  DoD  C/SCSC 
requirements . 

The  questionnaires  are  an  essential  step  In  a  DoD-sponsored 
study  effort  that  Arthur  D.  Little,  Inc.  Is  conducting  under  a 
contract  for  us.  Our  overall  study  effort  has  three  objectives: 

1.  Determine  the  degree  of  acceptance  and  use  of  the  C/SCSC. 

2.  Identify  problems  and  issues,  the  resolution  of  which 
could  lead  to  improvements  in  the  C/SCSC  and  associated  contract 
performance  reporting  requirements. 

3.  Recommend  policy  changes  that  would  lead  to  these 
improvements,  and  that  I  could  implement. 

I'd  like  your  help  in  accomplishing  these  objectives  by 
ensuring  that  the  questionnaires  are  completed  and  returned  no 
later  than  June  24,  1983.  The  questionnaires  have  been  designed 
to  be  brief  and,  above  all,  to  ensure  that  your  anonymity  will  be 
preserved.  Your  candor  and  cooperation  are  appreciated. 


Thank  you, 


Enclosure 
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\  OptnM 


Opinion  Rotoorch  Corporation 


600  Maryland  Avanu*.  S.W .  WMhington  0  C  2002' 


1202)  464  5992 


June  9,  1983 


Dear  Government  Program  Manager: 

Enclosed  are  two  types  of  questionnaires  relating  to  the  experience  of  your 
Program  Office  with  the  COST/SCHEDULE  CONTROL  SYSTEMS  CRITERIA  (C/SCSC).  The 
questionnaire  marked  Program  Manager  contains  questions  of  a  general  nature 
concerning  the  opinions  of'  you,  tne  Program  Manager,  toward  C/SCSC.  It 
should  take  only  a  few  minutes  of  your  time  to  fill  it  out. 


The  second  type  of  questionnaire,  marked  Contract  Specific,  contains  more 
detailed  questions  concerning  the  experience  of  your  office  with  C/SCSC  on  a 
specific  contract  with  a  specific  contractor.  We  would  appreciate  it  if  you 
would  have  the  person  in  your  Program  Office  who  is  in  charge  of  your 
C/SCSC-related  activities  complete  one  questionnaire  for  each  contractor 
having  a  C/SCSC  requirement. 

P lease  return  the  completed  questionnaires  in  the  enclosed  postage-paid 
envelope  by  June  24,  Ig83.  C 

We  appreciate  your  cooperation  in  this  important  survey  and  want  to  emphasize 
that  your  anonymity  will  be  preserved.  The  number  on  the  questionnaire  will 
be  used  to  tell  us  to  remove  your  name  from  our  mailing  list  when  we  receive 
your  questionnaires. 

If  you  have  any  questions  about  this  survey,  please  feel  free  to  call  me 
at  (609)  924-5900  weekdays,  8:30  A.M.  -  5:00  P.M. 


Sincerely, 


Linda  B.  Donnelly 
Research  Director 
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GOVERNMENT  PROGRAM  MANAGER 
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6Mb"  W.  <57(14-01 8i' 
Expires  12/31/83 


THE  COST/SCHEDULE  CONTROL  SYSTEMS  CRITERIA  ANALYSIS  STUDY 


The  first  questions  pertain  to  your  C/SCSC  program  in  general. 


1.  Please  indicate  the  Department  responsible  for  this  program. 


Check 

only 

one 


1  Q  Army 

2  □  Navy 

3  ^  Air  Force 


2.  Please  indicate  the  phase  of  this  program. 

_  ,  lG  Research  and  Development 

CheaK 

only  2Q  Production 

one 

3  [U  Other 


3.  Please  indicate  the  category  of  weapon/equipment  system  which  best 
describes  this  program. 

1  Q  Ai rcraft 

2  Q  Electronics 

_  .  3  Q  Missile 

Check 

only  4  Q  Ordnance 

one 

5  G Ship 

6  [j  Space 

7  CD  Surface  Vehicle 

8  Q  Other  (Please  specify):  _ 
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4.  What  will  be  the  approximate  total  cost  of  the  program  through  final 
production? 

1  O  Less  than  $25M 

2  D  At  least  525M  but  less  than  5100M 

Check  3  □  At  least  S100M  but  less  than  S500M 

only 

4  D  At  least  S500M  but  less  than  SIB 

5  0  At  least  SIB  but  less  than  S5B 

6  □  S5B  or  more 


5. 


If  you  had  the  choice  of  whether  or  not  you  would  require  your  contrac¬ 
tors)  to  have  a  C/SCSC-accepted  system(s),  would  you: 


Check 

only 

one 


1  f~1  Continue  to  require  the  currently  accepted  cost  and 
schedule  control  system(s)  as  operating  now 

2^3  Require  the  current  system(s)  with  minor  modifications 

3  n  Require  the  current  system(s)  with  major  modifications 


4  Q]  Not  require  the  current  system(s)  at  all? 


The  following  series  of  questions  deal  with  Cost  Performance  Reports  and 
your  opinions  concerning  CPRs. 


6.  Co  you  regularly  use  other  reports  or  other  means  in  lieu  of  using  Cost 
Performance  Reports  (CPRs)  or  data  derived  from  CPRs  in  oraer  to  deter¬ 
mine  the  cost  and  schedule  status  of  your  contract(s)? 

Check  1 33  Yes 

one  , _ _ 

—  2 r,c 
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7.  Please  rate  the  Cost  Performance  Reports  (CPRs)  or  data  directly  derived 
from  CPRs  in  terms  of  how  well  they  help  in  doing  each  of  the  following: 


Cheek  one  box  for  eaah  item 


Excellent 

Good 

Fair 

Poor 

No  Opinion 

Providing  an  indicator 
of  contract  schedule 
status 

in 

2D 

3D 

4D 

5D 

Determining  contract 
cost  status 

i  □ 

2D 

3  D 

4  D 

5  D 

Identifying  problem 
areas  not  previously 
recognized 

iD 

2D 

3D 

4  D 

sD 

Providing  the  caoabil- 
ity  of  tracing  problems 
to  their  sources 

iD 

2D 

3D 

4  D 

5D 

Identifying  cost 
impacts  of  known 
problems 

iG 

2D 

3  D 

4  D 

5  D 

Estimating  costs  to 
complete 

!□ 

2D 

3D 

4  D 

5  D 

8.  How  often  do  you  personally  review  the  cost  and  schedule  status  of  your 
contract(s)  based  on  Cost  Performance  Reports  (CPRs)  or  data  derived 
from  CPRs? 

1 1  1  More  often  than  monthly 
2  Monthly 

Cheek  3  Pj  Bimonthly 

only 

one  4  [_J  Quarterly 

5  l_i  Less  often  than  quarterly 

6  '_|  Never 
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9.  How  often  do  you  use  information  or  data  derived  from  CPRs  in  either 
briefings  or  written  reports  to  higher  headquarters  on  the  cost  and 
schedule  status  of  your  program? 


Check 

only 

one 


1  0  More  often  than  monthly 

2  O  Monthly 

3  0  Bimonthly 

4  0  Quarterly 

5  0  Less  often  than  quarterly 

6  0  Never 


Please  indicate  whether  you  agree,  disagree,  or  have  no  opinion  concerning 
the  statements  in  items  10  through  17: 


10.  The  concept  of  using  criteria  on  what  is  an  acceptable  cost  and  sched 
ule  control  system  rather  than  specifying  a  single  system  for  all  con 
tractors  to  use  is  not  valid: 

1  0  Agree  strongly 

2  0  Agree  somewhat 

Check 

only  3  0  Disagree  somewhat 
one  _ _ 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 


11.  The  C/SCSC  approach  of  ensuring  an  adequate  basis  for  keeping  the 
Program  Office  informed  on  the  cost  and  schedule  status  of  a  con¬ 
tract  is  as  good  or  better  than  other  approaches  to  achieving  tne 
same  objectives: 


Check 

only 

one 


1  0  Agree  strongly 

2  0  Agree  somewhat 

3  0  Disagree  somewhat 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 
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12.  The  specific  Cost  and  Schedule  Control  System  criteria  as  currently 
delineated  by  DoD  are  not  appropriate  for  evaluating  whether  a 
contractor's  system  provides  an  adequate  basis  for  properly  indicating 
cost  and  schedule  status. 

1  Fj  Agree  strongly 

2  G  Agree  somewhat 

Check  ~  ' 

only  3  Q  Disagree  somewhat 

one 

4  [j  Disagree  strongly 

5  G  Neither  agree  nor  disagree 


13.  A  major  benefit  of  C/SCSC  is  that  it  forces  a  contractor  to  do  planning 
that  otherwise  would  not  have  been  accomplished  as  thoroughly. 

1  G  Agree  strongly 


Check 

only 

one 


2  _ i  Agree  somewhat 

3  i_j  Disagree  somewhat 

4  G  Disagree  strongly 

5  G  Neither  agree  nor  disagree 


14.  DoD  C/SCSC  practices  are  not  effective  in  ensuring  that  an  adequate 
basis  exists  in  a  contractor's  system  for  reporting  cost  and  schedule 
status  to  the  program  manager. 


Check 

one 


1  G  Agree  strongly 

2  G  Agree  somewhat 

3  G  Disagree  somewhat 

4  G  Disagree  strongly 


5  j _ ,  Neither  agree  nor  disagree 
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15.  Cost  Performance  Reports  are  received  on  a  sufficiently  timely  basis  so 
as  to  be  useful  to  DoD  Program  Office  in  determining  cost  and/or  sched¬ 
ule  status: 


Check 

only 

one 


1  ; _ 1  Agree  strongly 

2  Zj  Agree  somewhat 

3  Zj  Disagree  somewhat 

4  0  Disagree  strongly 

5  Zj  Neither  agree  nor  disagree 


16.  The  costs  to  the  Program  Office  in  terms  of  time  and  manpower  expended 
for  all  C/SCSC-related  activities  are  outweighed  by  the  benefits  de¬ 
rived  by  the  Program  Office  in  managing  the  contract. 


Tneck 

only 

one 


1  (Zj  Agree  strongly 

2  ZZ  Agree  somewhat 

3  ZZ  Disagree  somewhat 

4  Zj  Disagree  strongly 

5  ZJ  Neither  agree  nor  disagree 


17.  Non-Program  Office  plant  representatives  such  as  DCAs,  DCAA,  NAVPROs, 
etc.  do  a  reasonably  good  job  of  monitoring  the  contractor's  C/SCSC- 
accepted  system  to  ensure  that  it  is  functioning  effectively. 


Check 

cr.Z'j 


1  Agree  strongly 

2  Zj  Agree  somewhat 

3  Disagree  somewhat 

4  ' _ .Disagree  strongly 

5  ZZ  Neither  agree  nor  disagree 
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18.  On  the  scale  below,  please  circle  one  number  for  each  item  to  indicate 
how  knowledgeable  you  personally  are  about  the  contents  of  each  of  the 
following  Cost  Performance  Report  (CPR)  Formats: 

1  =  Not  knowledgeable 

10  =  Very  knowledgeable 


a. 

Format  1  -  CWBS  levels 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

b. 

Format  2’  -  Organization 
levels 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

c. 

Format  3  -  Baseline 
information 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

d. 

Format  4  -  Manpower 
i nformation 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

e. 

Format  5  -  Problem 
analysi s 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Now,  just  a  few  questions  about  yourself  so  that  your  responses  can  be  com¬ 
pared  with  those  of  other  individuals  with  experience  similar  to  yours. 

19.  How  long  have  you  been  in  your  current  job? 

1  rj  Less  than  one  year 

Check 

only  2  []  1  to  3  years 

one 

3  C  More  than  3  years 

20.  Did  you  have  a  previous  job  which  provided  you  with  some  familiarity 
with  C/SCSC? 

Check  1  □  Yes 

one 

2  , _ i  No 

21.  What  C/SCSC  training,  if  any,  have  you  had? 

1  [j  C/SCSC-related  courses  at  Defense  Systems  Management 
College  (DSMC),  Air  Force  Institute  of  Technology,  or 

Check  Army  Management  Engineering  Training  Agency 

ohuxt  2  I  i  Other  formal  training  (e..g..  Industry  Seminar,  portion 

jt r ly  of  DSCM  course,  etc.) 

3  In-house  or  On-the-job  training 

4  None 
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22.  Finally,  please  give  us  your  thoughts  on  what  steps  DoD  might  take  to 
improve  the  usefulness  of  C/SCSC  to  you  and/or  to  your  contractor(s) . 

If  you  wish  to  comment  about  any  other  aspect  of  C/SCSC,  we  welcome  such 
observations.  These  comments,  as  well  as  all  other  information  provided 
in  this  questionnaire,  will  remain  completely  confidential .  Your 
responses  will  never  be  associated  with  any  information  by  which  you  can 
be  personally  identified.  If  you  choose  to  respond  --  and  we  hope  you 
will  --  please  use  the  space  below  and  feel  free  to  add  extra  pages  as 
necessary.  We  would  appreciate  hearing  from  you. 
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CONTRACT  SPECIFIC 


June  9,  1983 


Dear  Government  Business  Managers: 

Please  complete  one  of  these  questionnaires  for  each  of  your  contractors 
having  a  COST/SCHEDULE  CONTROL  SYSTEMS  CRITERIA  (C/SCSC)  requirement.  If  a 
contractor  has  more  than  one  active  contract  with  a  C/SCSC  requirement,  then 
please  select  the  active  contract  that  most  recently  has  been  through  a 
Demonstration  Review  and  resulted  in  DOD  acceptance.  If  you  have  no  active 
contract  meeting  this  criterion,  then  select  the  most  recent  active  contract 
to  have  completed  the  C/SCSC  Subsequent  Application  Review  (SAR)  process. 

If  you  have  no  active  contracts  meeting  the  above  criteria,  then  select  the 
most  recently  completed  contract  that  went  through  the  Demonstration  Review 
process  and  resulted  in  DOD  acceptance.  If  you  do  not  have  such  a  completed 
contract,  then  select  the  most  recently  completed  contract  that  has  completed 
the  SAR  process. 

If  there  are  no  contracts  with  a  specific  contractor  that  meet  these  cri¬ 
teria,  then  for  that  specific  contractor  answer  only  question  33  and  include 
the  comment:  "No  contract  for  this  contractor  and  program  has  completed  the 
C/SCSC  review  process." 

Please  answer  all  questions  in  this  questionnaire  in  terms  of  the  specific 
contracts )  you  have  selected  for  this  questionnaire.  We  do  not  need  to 
know  the  contract  name. 

The  questionnaire  should  be  returned  with  the  completed  Government  Program 
Manager's  questionnaire  by  June  24,  1983. 

We  appreciate  your  cooperation  in  this  important  survey  and  want  to  emphasize 
that  your  anonymity  will  be  preserved.  The  number  on  the  questionnaire  will 
be  used  to  tell  us  to  remove  you r  name  from  our  mailing  list  when  we  receive 
your  questionnaires. 
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GOVERNMENT  BUSINESS  MANAGERS 


CONTRACT  SPECIFIC 


OMB  NO.  0704-0183 
Expires  12/31/83 


THE  COST/SCHEOULE  CONTROL  SYSTEMS  CRITERIA  ANALYSIS  STUDY 


The  first  questions  pertain  to  this  contract  in  general. 

1.  The  contract  selected  uses  a  C/SCSC-compl iant  system  which  was  accepted 
by  means  of  .  .  . 

1  j  |  Demonstration  Review 

2  n  Subsequent  Application  Review  (SAR) 

indicate  the  Department  which  awarded  this  contract. 

1  Q  Army 

2  □  Navy 

3  Lj  Air  Force 

indicate  the  predominant  type  of  work  being  done  under  this 
contract. 

1  Advanced  development 

2  D  Engineering  development 

Check 

only  3  Q  Low  rate  production 

one 

4  |  | Production 

5  □  Other 


Check 

one 


2.  Please 

Check 

only 

one 


3.  Please 
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4.  Please  indicate  the  category  of  weapon/ equipment  system  which  best 
describes  this  contract: 

1  CD  Aircraft 

2  CD  Electronics 

3  □  Missile 

Check  i — | 

only  4  LJ  Ordnance 

one  , — , 

-  5  □  Ship 

6  CD  Space 

7  Q  Surface  vehicle 

8  Q  Other  (Please  specify):  _ 


5.  What  is  the  approximate  total  dollar  value  of  this  contract? 

1  □  Under  $10M 

2  □  At  least  S1CM  but  less  than  S25M 

3  CD  At  least  $25M  but  less  than  S40M 

Check  ^ 

4  □  At  least  $40M  but  less  than  S100M 

one 

5  □  At  least  S10CM  but  less  than  S160M 

6  CD  At  least  S160M  but  less  than  S500M 

7  Cj  J500M  or  more 


6. 


What  type  of  contract  is  the  one  you  have  selected? 


Check 

only 

one 


1  □  CPFF 

2  CDcpaf 

3  □  CPIF 

4  □  FPI 

5  CD  Other  (Please  specify): 
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7.  What  is  the  duration  of  the  contract? 


Check 

only 

one 


1  O  Less  than  12  months 

2  Q  At  least  12  months  but  less  than  24  months 

3  Q  At  least  24  months  but  less  than  36  months 

4  Q  At  least  36  months  but  less  than  48  months 

5  Q  At  least  48  months  but  less  than  60  months 

6  Q  60  months  or  more 


Was  the  contract  definitized  at  the  time  of  contract  award? 

Check  1  □  Yes - ►SKIP  TO  Q. 10 

one 

s2  □  No 

X 

IF  "NO" 

9.  Approximately  how  many  months  elapsed  from  contract  award  to 
contract  definitizatton? 


Check 

only 

one 


1  Q  Less  than  3  months 

2  Q  At  least  3  months  but  less  than  6 

3  Q  At  least  6  months  but  less  than  9 

4  Q  At  least  9  months  but  less  than  12 

5  O  At  least  12  months  but  less  than  15 

6  Q  At  least  15  months  but  less  than  18 

7  Q  18  months  or  more 


10.  Approximately  how  many  months  after  the  contract  award  did  the  contrac¬ 
tor  establish  the  C/SCSC  performance  measurement  baseline? 

1  CD  Less  than  3  months 

2  Q  At  least  3  months  but  less  than  6 

3  Q  At  least  6  months  but  less  than  9 

Check 

only  4  D  At  least  9  months  but  less  than  12 

one 

5  CD  At  least  12  months  but  less  than  15 

6  CD  At  least  15  months  but  less  than  18 

7  D  At  least  18  months  but  less  than  21 

8  D  At  least  21  months  but  less  than  24 

9  CD  24  months  or  more 


11.  Approximately  how  many  months  after  contract  award  would  have  been  the 
earliest  you  could  reasonably  have  expected  the  C/SCSC  performance 
measurement  baseline  to  have  been  established  by  the  contractor? 

1  CD  Less  than  3  months 

2  CD  At  least  3  months  but  less  than  6 

3  CD  At  least  6  months  but  less  than  9 

Check 

only  4  CD  At  least  9  months  but  less  than  12 

one  __ 

5  CD  At  least  12  months  but  less  than  15 

6  CD  At  least  15  months  but  less  than  18 

7  CD  At  least  18  months  but  less  than  21 

8  CD  At  least  21  months  but  less  than  24 

9  CD  24  months  or  more 
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The  next  series  of  questions  deal  with  the  DoD  C/SCSC  review  process  that 
normally  entails  several  in-plant  visits  by  DoD  C/SCSC  review  teams. 

The  Question  12  series  should  be  answered  in  reference  only  to  the  one  visit 
that  you  consider  to  have  been  the  key  DoD  C/SCSC  review  for  this  contract. 


Based  on  your  recollection  of  tine  key  review  .  .  . 


12a.  Approximately  how  many  months  after  contract  award  was  the  Demonstra¬ 
tion  Review  or  the  Subsequent  Application  Review  held? 

1  Q  Less  than  3  months 

2  At  least  3  months  but  less  than  6 

3  O  At  least  6  months  but  less  than  12 

4  Q  At  least  12  months  but  less  than  18 

5  []  At  least  18  months  but  less  than  24 

6  O  At  least  24  months  but  less  than  30 

7  C  At  least  30  months  but  less  than  36 

8  P]  36  months  or  more 


Check 

only 

one 


12b.  Did  Program  Office  representative(s)  participate  in  the  review? 
,1  □  Yes 

2  □  No - »>  SKIP  TO  Q.  13 


IF  "YES” _ 

j  12c.  What  organizations  were  represented? 

1  1  Q  Program  Office 

'  2  PI  Service  C/SCSC  Focal  Point,  i.e.,  HQ  DARCOM, 

;  Check  HQ  NAVMAT,  HQ  AFSC 

■  ill 

1  that  3  n  Government  plant  representatives,  e.g.,  DCAS, 
j  arply  AFPRO,  NAVPRO,  etc. 

4Q  Commodity  Command,  e.g.,  MICOM,  ASD,  NAVAIR,  etc. 

5  □  DCAA 

6  [J  Other  (Please  specify):  _ 
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12d.  What  was  the  duration  of  the  review  in  workdays? 

1  Q  Under  3  workdays 

2  Q  3-5  workdays 

Check 

only  3  □  6  -10  workdays 

cm 

4  Q  11-15  workdays 

5  n  More  than  15  workdays 

12e.  What  was  the  size  of  the  Review  Team? 

Less  than  4  persons 
4-7  persons 
8-11  persons 
12-15  persons 
16-19  persons 
20-23  persons 
24  or  more  persons 


iD 
20 
3  □ 

Check  4  |  ( 

only 

£2£  5  □ 

6  □ 
;□ 


12f.  Approximately  how  many  discrepancies  or  corrective  action  items  were 
identified  by  the  DoO  Review  Team? 

1  Q  10  or  less 

2  □  11-20 

Check  3  Q  21-30 

only 

£21  4  Q  31-40 

5  □  41-50 

6  □  More  than  50 

7  Q  Don't  know,  too  difficult  to  aporoximate 
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13.  In  what  year  did  this  contractor  receive  official  notification  that  the 
C/SCSC  requirement  had  been  satisfactorily  met  for  this  contract? 


Check 

only 

one 


1  □  1983 

2  □  1982 

3  □  1981 

4  Qigso 

5  □  1979 

6  □  1978 

7  □  1977 

8  □  1976 

9  0  1975  or  earl ier 


Now,  we  would  like  your  opinion  on  various  aspects  of  the  review  process. 

14.  In  your  view,  how  necessary  is  the  participation  of  the  following  to 
the  C/SCSC  review  process  .  .  .? 


Check  one  box  for  each  item 


Very 

Necessary 

Somewha  t 
Necessary 

Not 

Necessary 

a. 

Commodity  Command  participation, 
e.g.,  ASD,  NAVAIR,  MICOM,  etc. 

1  □ 

2  □ 

3  □ 

b. 

Service  C/SCSC  Focal  Point 
participation,  i.e.,  HQ  DARCOM, 
HQ  NAVMAT,  HQ  AFSC 

1  □ 

2  □ 

3  □ 

c. 

Contract  Administration  Office 
participation,  e.g.,  DCAS, 

AFPRO,  etc. 

1  □ 

2  □ 

3  □ 

d. 

Defense  Contract  Audit  Agency 
(DCAA)  participation 

iD 

2  □ 

3  □ 

e. 

Program  Office  participation 

iD 

zD 

3  □ 
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15.  All  things  considered,  how  necessary  is  the  entire  C/SCSC  review  process 
itself  to  the  proper  functioning  of  the  contractor's  cost  and  schedule 
control  system? 

1  □  Very  necessary 

Checx 

°nlu  2  □  Somewhat  necessary 

one 

3  0  Not  necessary 

The  following  series  of  questions  deal  with  Cost  Performance  Reports  and 
your  opinions  concerning  CPRs. 


16.  Considering  the  contract  Itself  as  CWBS  Level  1,  what  is  the  lowest  CWBS 
level  regularly  reported  to  you  by  the  contractor  on  Format  1  of  the 
Cost  Performance  Report? 


1  Q  Level 

2 

2  0  Level 

3 

Check  • 

3  0  Level 

4 

only 

one 

4  0  Level 

5 

5  0  Level 

6 

6  0  Level 

7 

7  0  Level 

8 

8  0  Level 

9  or  lower 

17.  For  meaningful  contract  analysis,  this  CWBS  level,  in  your  opinion,  is: 


1  0  Too  low 
2D  Too  high 

-  3  1 _ I  Just  about  right 

4  O  No  opinion 

18.  Which  of  the  statements  below  best  characterizes  the  extent  to  which  the 
Program  Office  routinely  reviews  and  analyzes  the  Cost  Performance 
Reports  received  each  month? 

1  0  We  review  and/or  analyze  the  report  in  its  entirety 

2  Tj  We  review  and/or  analyze  only  certain  Formats 


3  0  We  review  and/or  analyze  only  those  portions  of  the 

report  where  a  variance  threshold  has  been  exceeded 

4  0  We  do  not  review  and/or  analyze  the  reports 


c 


19.  In  your  opinion,  could  the  number  of  pages  in  the  Cost  Performance 
Reports  you  receive  be  reduced  without  hampering  your  ability  to 
manage  the  contract? 


Check  . 1  □  Yes 


?ne 


X 

IF  "YES” 


2  □  No 


>  SKIP  TO  Q . 22 


20^  Are  all  or  some  of  the  pages  that  could  be  eliminated  submitted 
solely  at  the  contractor's  choice  (e.g.,  internal  computer 
printouts  in  lieu  of  summary  data,  etc.)? 


lD  All - ►  SKIP  TO  Q.22 


Check 

only 

one 


2  □  Some 

Aip  None 
IF  "SOME"  OR  "NONE 


II 


21.  For  the  pages  that  are  not  submitted  solely  at  the 
contractor's  choice,  please  give  your  rough  estimate 
of  the  percentage  of  pages  that  could  be  reduced. 


Check 

only 

one 


□ 

1  - 

5 

□ 

6  - 

10 

□ 

11  - 

20 

□ 

21  - 

30 

□ 

31  - 

40 

□ 

41  - 

50 

□ 

51  - 

75 

□ 

Over 

75 

percent 

percent 

percent 

percent 

percent 

percent 

percent 

percent 


22.  Approximately  how  many  calendar  weeks  after  the  close  of  the 
contractor's  reporting  period  are  Cost  Performance  Reports 
received  by  the  Program  Office? 


1  D  Less  than  one  week 

2  O  At  least  1  week  but  less  than  2 

3  D  At  least  2  weeks  but  less  than  3 

4  CH  At  least  3  weeks  but  less  than  4 

5  □  At  least  4  weeks  but  less  than  5 

5  CU  5  weeks  or  more 
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23.  On  average,  how  many  calendar  weeks  elapse  from  the  time  Cost 
Performance  Reports  are  received  by  the  Program  Office  until 
reviews  and  analyses  are  completed  within  the  Program  Office? 

1  □  Less  than  1  week 

2  D  At  least  1  week  but  less  than  2 
Cheek  3  D  At  least  2  weeks  but  less  than  3 

OKu'J  | - | 

one  4  1 _ I  At  least  3  weeks  but  less  than  4 

5  D  At  least  4  weeks  but  less  than  5 

6  □  5  weeks  or  more 


24.  Approximately  how  many  Program  Office  man-days  (include  all  Program 
Office  personnel  and  contracted  support,  if  any)  on  average  are  spent 
per  month  collecting,  reviewing,  analyzing,  and  summarizing  information 
contained  in  contractor  Cost  Performance  Reports? 

1  Q  Less  than  1 

2  □  1-4 

Cheek  3  □  5-9 

:ne  4  U  10-14 

5  □  15-19 

6  □  20-24 

7  □  25-29 

80  30  or  more 
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25.  Please  rate  the  Cost  Performance  Reports  (CPRs)  in  terms  of  how  well 
they  help  in  doing  each  of  the  following: 


Check  one  box  for  each  item 


Excel  1 ent 

Good 

Fai  r 

Poor 

No  Opinion 

a. 

Providing  an  indicator 
of  contract  schedule 

zU 

3  □ 

4  ED 

5  □ 

status 

1  □ 

b. 

Determining  contract 
cost  status 

iD 

zD 

3ed 

4  □ 

50 

c. 

Identifying  problem 
areas  not  previously 
recognized 

iD 

zD 

3  □ 

4d 

50 

d. 

Providing  the  capabil- 
i ti ty  of  tracing 
problems  to  their 
sources 

iED 

z  □ 

3  □ 

4  Li 

50 

e. 

Identifying  cost  impacts 
of  known  problems 

i  □ 

2  □ 

30 

4  □ 

5  O 

f. 

Estimating  costs  to 
complete 

iD 

zD 

30 

4  □ 

sD 

Please  indicate  whether  you  agree,  disagree,  or  have  no  opinion  concerning 
the  statements  in  items  26  through  32: 


26.  The  contractor's  cost  and  schedule  control  systems  and  procedures  used 
for  this  contract  are  effective  in  helping  management  to  control  con¬ 
tract  performance. 

1  EH  Agree  strongly 

2  CH  Agree  somewhat 

Check 

only  3  ED  Disagree  somewhat 
one 

4  I _ I  Disagree  strongly 

5  1 _ i  Neither  agree  nor  disagree 
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27.  The  specific  cost  and  schedule  control  systems  cri ten  a  as  currently 
delineated  by  DoO  are  not  appropriate  for  evaluating  whether  a 
contractor's  system  provides  an  adequate  basis  for  properly  indicating 
cost  and  schedule  status  on  this  type  of  contract. 


1  □  Agree  strongly 

2  EH  Agree  somewhat 

3  I _ !  Disagree  somewhat 

4  CD  Disagree  strongly 

5  □  Neither  agree  nor  disagree 


28.  The  contractor's  cost  schedule  and  control  system  provides  internal  data 
for  this  contract  which  indicate  work  progress,  properly  relate  cost, 
schedule  and  technical  accomplishment,  are  valid,  timely  and  auditable, 
and  supply  the  Program  Office  with  information  at  a  practicable  level  of 
summarization. 

1  □  Agree  strongly 

2  Q  Agree  somewhat 

Z'<’.sa<  _ 

-riy  3  O  Disagree  somewhat 

SKfS 

4  □  Disagree  strongly 

5  O  Neither  agree  nor  disagree 


29.  DoD  C/SCSC  practices  on  this  contract  were  not  effective  in  ensuring 
that  an  adequate  basis  existed  in  the  contractor's  system  for 
reporting  cost  and  schedule  status  to  the  program  manager. 


1  O  Agree  strongly 

2  □  Agree  somewhat 

3  □  Disagree  somewhat 

4  Eli  Disagree  strongly 

5  Q  Neither  agree  nor  disagree 
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30.  The  Cost  Performance  Report  is  received  on  a  sufficiently  timely  basis 
so  as  to  be  useful  to  the  Program  Office  in  determining  cost  and/or 
schedule  status: 


Check 

only 

one 


1  CD  Agree  strongly 

2  CD  Agree  somewhat 

3  CD  Disagree  somewhat 

4  D  Disagree  strongly 

5  □  Neither  agree  nor  disagree 


31.  The  costs  to  the  Program  Office  in  terms  of  time  and  manpower  expended 
for  all  C/SCSC-related  activities  are  outweighed  by  the  benefits  de¬ 
rived  by  the  Program  Office  in  managing  this  contract. 

1  EH  Agree  strongly 
Check  2  D  A9ree  somewhat 

°0ngC'  3  CD  Disagree  somewhat 

4  ED  Disagree  strongly 

5  CD  Neither  agree  nor  disagree 


32.  Non-Program  Office  plant  representatives  such  as  DCAS,  DCAA,  NAVPROs, 
etc.  do  a  reasonably  good  job  of  monitoring  the  contractor's  C/SCSC- 
accepted  system  to  ensure  that  it  is  functioning  effectively. 

1  CD  Agree  strongly 

2  CD  Agree  somewhat 

3  CD  Disagree  somewhat 

4  CD  Disagree  strongly 

5  ED  Neither  agree  nor  disagree 


Tneck 

cnl'j 
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Finally,  please  give  us  your  thoughts  on  what  steps  DoD  might  take  to 
improve  the  usefulness  of  C/SCSC  to  you  and/or  to  this  contractor.  If 
you  wish  to  comment  about  any  other  aspect  of  C/SCSC,  we  welcome  such 
observations.  These  comments,  as  well  as  all  other  information  provided 
in  this  questionnaire,  will  remain  completely  confidential.  Your 
responses  will  never  be  associated  with  any  information  by  which  you  can 
be  personally  identified.  If  you  choose  to  respond  --  and  we  hope  you 
will  --  please  use  the  space  below  and  feel  free  to  add  extra  pages  as 
necessary.  We  would  appreciate  hearing  from  you. 
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600  Maryland  Avenue  S  W  Washington  0  C  20024 

>202)  4M.5992 


June  9,  1983 


Dear  Contractor  Program  Manager: 

Enclosed  are  two  types  of  questionnaires  relating  to  the  experience  of  your 
Program  Office  with  the  COST/SCHEDULE  CONTROL  SYSTEMS  CRITERIA  (C/SCSC).  The 
questionnaire  marked  Program  Manager  contains  questions  of  a  general  nature 
concerning  the  opinions  of  you,  tne  Program  Manager,  toward  C/SCSC.  It 
should  take  only  a  few  minutes  of  your  time  to  fill  it  out. 

The  second  questionnaire,  marked  Contract  Specific,  contains  more  detailed 
questions  concerning  the  experience  of  your  office  with  C/SCSC  on  a  specific 
contract.  We  would  appreciate  it  if  you  would  have  the  person  in  your 
Program  Office  who  is  in  charge  of  your  C/SCSC-related  activities  complete 
the  second  questionnaire. 

Please  return  both  completed  questionnaires  in  the  enclosed  postage-paid 
envelope  by  June  24,  1 9&5 1 

We  appreciate  your  cooperation  in  this  important  survey  and  want  to  emphasize 
that  your  anonymity  will  be  preserved.  The  number  on  the  questionnaire  will 
be  used  to  tell  us  to  remove  your  name  from  our  mailing  list  when  we  receive 
your  questionnaires. 

If  you  have  any  questions  about  this  survey,  please  feel  free  to  call  me 
at  (609)  924-5900  weekdays,  8:30  A.M.  -  5:00  P.M. 

Sincerely, 

5*  <0^ 


Linda  B.  Donnelly 
Research  Director 
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QMB  NO.  0704-0183 
Expires  12/31/8? 

CONTRACTOR  PROGRAM  MANAGER  | 


THE  COST/SCHEDULE  CONTROL  SYSTEMS  CRITERIA  ANALYSIS  STUDY 


The  first  questions  pertain  to  your  C/SCSC  program  in  general. 

1.  Please  indicate  the  military  department  responsible  for  this  program. 

lH  Army 
Check  _ 

only  2Hj  Navy 

one 

3l  1  Air  Force 

2.  Please  indicate  the  phase  of  this  program. 

in  Research  and  Development 

Check 

only  2Q  Production 

one 

3Q  Other 

3.  Please  indicate  the  category  of  weapon/equipment  system  which  best 
describes  this  program. 

in  Aircraft 

2Hj  Electronics 

3Q  Missile 

Check 

only  4Q  Ordnance 

one  __ 

5 G  Ship 

6[j  Space 

7  Surface  Vehicle 

8^1  Other  (Please  specify):  _ 
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4.  What  will  be  the  approximate  total  cost  of  the  program  through  final 
production? 


lHj  Less  than  S25M 

2G  At  least  S25M  but  less  than  $100M 

Check 

only  3  [3  At  least  S100M  but  less  than  S500M 
one  __ 

4  [j  At  least  5500M  but  less  than  SIB 

5  O  At  least  $  IB  but  less  than  S5B 

6  [j  S5B  or  more 


5.  If  you  had  the  choice  of  whether  or  not  you  would  use  your  C/SCSC- 
accepted  system  to  assist  you  in  managing  your  program,  would  you: 


Cneak 

only 

one 


1 _ 1  Continue  to  use  the  C/SCSC-accepted  system  as  it  is 

currently  operating  now 

2 r~i  Use  your  C/SCSC-accepted  system  with  minor  modifications 

3 Use  your  C/SCSC-accepted  system  with  major  modifications 

4 H  Not  use  the  C/SCSC-accepted  system  at  all? 


The  following  series  of  questions  deal  with  reporting. 


6.  Do  you  regularly  use  other  reports  or  other  means  in  lieu  of  using 

C/SCSC-driven  internal  cost  and  schedule  reports  in  order  to  determine 
cost  and  schedule  status  of  your  contract(s)? 

Check  IC  ^es 

one  — 

2U  No 
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Please  rate  your  C/SCSC-driven  internal  cost  and  schedule  reports  in 
terms  of  how  well  they  help  in  doing  each  of  the  following: 


Check  one  box  for  each  item 


Excellent 

Good 

Fair 

Poor 

No  Opi 

a. 

Providing  an  indicator 
of  contract  schedule 

1  iZj 

3G 

status 

2  □ 

5U 

b. 

Determining  contract 
cost  status 

iD 

20 

3G 

4G 

5  G 

c. 

Identifying  problem 
areas  not  previously 

iG 

3U 

4G 

5G 

recognized 

2d 

d. 

Providing  the  capabil¬ 
ity  of  tracing  problems 
to  their  sources 

iD 

2d 

3G 

4G 

sG 

e. 

Identifying  cost 
impacts  of  known 
problems 

iG 

2d 

3  □ 

4  G 

5G 

f. 

Estimating  costs  to 
complete 

iD 

2d 

3G 

4  G 

5G 

How  often  do  you  personally  review  reports  of  any  type  either  generated 
directly  by  your  C/SCSC-accepted  system  or  derived  from  your  system? 

lG  Da^y 

2 r~]  Weekly 

Check 

only  3[j  Biweekly 
one 

4  G  Monthly 

5  Q  Quarterly 

6  [_j  Less  often  than  quarterly 

7  G  Never 
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The  following  questions  concern  your  use  of  the  C/SCSC-accepted  system  or 
internally  tailored  systems. 

9.  Please  indicate  the  response  that  best  describes  how  often  your  company 
uses  its  C/SCSC-accepted  system  on  DoD  contracts,  when  C/SCSC  is  not  a 
contractual  requirement. 

10  All  of  the  time 

Check  2  0  Some  of  the  time 

only 

one  3  0  Never 

4  0  Don't  know 


10.  Please  indicate  the  response  that  best  describes  how  often  your  company 
uses  an  internally  tailored  earned-value  system  on  DoD  contracts,  when 
earned-value  is  not  a  contractual  requirement. 

10  All  of  the  time 

Check  2  0  Some  of  the  time 

only 

one  3  0  Never 

4  0  Don't  know 

11.  Does  your  company  have  commercial  work? 


10  Yes 

fc0  no - ►SKIP  TO  Q.  14 


IF  "YES" 

12.  Plea'e  indicate  the  response  that  best  describes  how  often  your 
comp,  ny  uses  its  C/SCSC-accepted  system  on  commercial  work. 


i  .  !□  All  of  the  time 

,  Check 

: only  2  0  Some  of  the  time 

■one 

!  3  0  Never 

l 

4  0  Don ' t  know 

I 

13.  Please  indicate  the  response  that  best  describes  how  often  your 
!  company  uses  an  internally  tailored  earned-value  system  on 
;  commercial  work. 


j 

.  Theck 


10  All  of  the  time 

2  0  Some  of  the  time 

3  0  Never 

4  0  Don't  know 
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Please  indicate  whether  you  agree,  disagree,  or  have  no  opinion  concerning 
the  statements  in  items  14  through  20: 


14.  The  concept  of  using  criteria  on  what  is  an  acceptable  cost  and  sched 
ule  control  system  rather  than  specifying  a  single  system  for  all  con 
tractors  to  .use  is  not  valid: 

1 D  Agree  strongly 

2  [j  Agree  somewhat 

3  Q  Disagree  somewhat 

4  Q  Disagree  strongly 

5  Q  Neither  agree  nor  disagree 

15.  The  C/SCSC  approach  of  ensuring  an  adequate  basis  for  keeping  your 
Program  Office  informed  on  the  cost  and  schedule  status  of  a  con¬ 
tract  is  as  good  or  better  than  other  approaches  to  achieving  the 
same  objectives: 

lD  Agree  strongly 

2  D  Agree  somewhat 

3  LH  Disagree  somewhat 

4  □  Disagree  strongly 

5  Q  Neither  agree  nor  disagree 

16.  The  specific  Cost  and  Schedule  Control  System  criteria  as  currently 
delineated  by  DoD  are  not  appropriate  for  evaluating  whether  your 
company’s  system  provides  an  adequate  basis  for  properly  indicating 
cost  and  schedule  status. 

1  C  Agree  strongly 

2  [j  Agree  somewhat 

3  D  Disagree  somewhat 

4  □  Disagree  strongly 

5  C  Neither  agree  nor  disagree 


Check 

only 

one 


Check 

only 

one 


Check 

only 

one 
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17.  A  major  benefit  of  C/SCSC  is  that  it  forces  a  contractor  to  do  planning 
that  otherwise  would  not  have  been  accomplished  as  thoroughly. 


Check 

only 

one 


10  Agree  strongly 
2 0  Agree  somewhat 

3  0  Disagree  somewhat 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 


18.  DoD  C/SCSC  practices  are  not  effective  in  ensuring  than  an  adequate 
basis  exists  in  your  company's  system  for  reporting  cost  and  schedule 
status  to  the  program  manager. 


Check 

only 

one 


10  Agree  strongly 

2  0  Agree  somewhat 

3  0  Disagree  somewhat 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 


19.  C/SCSC-dri ven  internal  cost  and  schedule  reports  are  received  on  a  suf¬ 
ficiently  timely  basis  so  as  to  be  useful  to  your  Program  Office  in 
determining  cost  and/or  schedule  status. 

1 0  Agree  strongly 

rrneck  2  0  Agree  somewhat 

3  _ |  Disagree  somewhat 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 


20.  The  costs  to  your  company  (and  to  DoD)  in  terms  of  time  and  manpower 
expended  to  operate  the  C/SCSC-accepted  system  are  outweighed  by  the 
benefits  derived  by  your  Program  Office  in  managing  the  contract. 

1  0  Agree  strongly 

2  0  Agree  somewhat 

Cneck 

only  3  0  Disagree  somewhat 

4  _ '  Disagree  strongly 

5  0  Neither  agree  nor  disagree 
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21.  On  the  scale  below,  please  circle  one  number  for  each  item  to  indicate 
how  knowledgeable  you  personally  are  about  the  contents  of  each  of  the 
following  Cost  Performance  Report  (CPR)  Formats: 

1  =  Not  knowledgeable 

10  =  Very  knowledgeable 


a. 

Format  1  -  CWBS  levels 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

b. 

Format  2  -  Organization 
levels 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

c. 

Format  3  -  Baseline 
information 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

d. 

Format  4  -  Manpower 
information 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

e. 

Format  5  -  Problem 
analysis 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

22.  In  your  judgment,  how  effective  has  your  C/SCSC-accepted  system  been  in 
assisting  you  and  your  staff  to  keeping  your  program  on  schedule  and 
within  cost  limits? 

1  d  Very  effective 

Check  2  d  Somewhat  effective 

only 

one  3  LJ  Not  too  effective 

4  d  Not  at  all  effective 


23.  In  your  view,  how  accurately  (for  the  "as  of"  date  of  the  report)  do 
the  Cost  Performance  Reports  portray  the  cost  and  schedule  status  of 
the  contracts )  for  your  program? 

id  Very  accurately 

Check  2  d  Somewhat  accurately 

only 

one  3  LI  Not  too  accurately 

4  d  Not  at  all  accurately 
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Now,  just  a  few  questions  about  yourself  so  that  your  responses  can  be  com¬ 
pared  with  those  of  other  individuals  with  experience  similar  to  yours. 


24.  How  long  have  you  been  in  your  current  job? 


Check 

only 

one 


1  f~~)  Less  than  one  year 

2  [3  1  to  3  years 

3  More  than  3  years 


25. 


Oid  you  have  a 


with  C/SCSC? 


Check 

one 


HI 

23 


previous  job  which  provided  you  with  some  familiarity 

Yes 

No 


26.  What  C/SCSC  training,  if  any,  have  you  had? 


Cneck 

all 

that 

apply 


1  PI  C/SCSC-related  courses  at  Defense  Systems  Management 

College  (DSMC),  Air  Force  Institute  of  Technology,  or 
Army  Management  Engineering  Training  Agency 

2  r ;  Other  formal  training  (e. .g.,  Industry  Seminar,  portion 

of  DSCM  course,  etc.) 

3  In-house  or  On-the-job  training 
4 1  ' i  None 


27.  Finally,  please  give  us  your  thoughts  on  what  steps  DoD  might  take  to 
improve  the  usefulness  of  C/SCSC  to  you  ana/or  to  your  contractor(s) . 

If  you  wish  to  comment  about  any  other  aspect  of  C/SCSC,  we  welcome  such 
observations.  These  comments,  as  well  as  all  other  information  provided 
in  this  questionnaire,  will  remain  completely  confidential.  Your 
responses  will  never  be  associated  with  any  information  by  which  you  can 
be  personally  identified.  If  you  choose  to  respond  --  ana  we  hope  you 
will  --  please  use  the  space  below  and  feel  free  to  add  extra  pages  as 
necessary.  We  would  appreciate  hearing  from  you. 
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CONTRACT  SPECIFIC 


June  9,  1983 


Dear  Contractor  Business  Manager: 

The  enclosed  questionnaire  asks  for  detailed  information  about  one  specific 
contract  on  which  compliance  with  the  COST/SCHEDULE  CONTROL  SYSTEMS  CRITERIA 
(C/SCSC)  is  required.  If  you  have  more  than  one  active  contract  in  your  pro¬ 
gram,  please  select  the  most  recent  one  that  has  been  through  a  Demonstration 
Review  and  has  resulted  in  000  acceptance.  If  you  have  no  contract  meeting 
this  criterion,  then  select  the  most  recent  active  contract  to  have  completed 
the  C/SCSC  Subsequent  Application  Review  (SAR)  process. 

If  you  have  no  active  contracts  meeting  the  above  criteria,  then  select  the 
most  recently  completed  contract  that  went  through  the  Demonstration  Review 
process  and  resulted  in  DOD  acceptance.  If  you  do  not  have  such  a  completed 
contract,  then  select  the  most  recently  completed  contract  that  has  completed 
the  SAR  process. 

Please  answer  all  questions  in  this  questionnaire  in  terms  of  the  one  specific 
contract  you  have  selected.  We  do  not  need  to  know  the  contract  name. 

If  you  have  no  contracts  that  meet  these  criteria,  then  answer  only  question 
37  and  include  the  comment:  "No  contract  in  this  program  has  completed  the 
C/SCSC  review  process." 

This  questionnaire  should  be  returned  with  the  completed  Contractor  Program 
Manager's  questionnaire  by  June  24,  1983. 

We  appreciate  your  cooperation  in  this  important  survey  and  want  to  emphasize 
that  your  anonymity  will  be  preserved.  The  number  on  the  questionnaire  will 
be  used  to  tell  us  to  remove  your  name  from  our  mailing  list  when  we  receive 
your  questionnaires. 
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CONTRACTOR  BUSINESS  MANAGERS 


CONTRACT  SPECIFIC 


OMB  No.  0704-0183 
Expires  12/31/83 


THE  COST/SCHEDULE  CONTROL  SYSTEMS  CRITERIA  ANALYSIS  STUDY 


The  first  questions  pertain  to  the  selected  contract  in  general. 


1.  The  contract  selected  by  you  uses  an  operating  C/SCSC-compl iant  system 
which  was  accepted  by  DoD  by  means  of  .  .  . 

r-,iea-K  lQ  Demonstration  Review 

2Q  Subsequent  Application  Review  (SAR) 


Please  indicate  the  Military  Department  which  awarded  this  contract. 

1  [H  Army 

2  Q  Navy 

3  Ai  r  Force 


,neoK 

ml, 

S'ltz 


Please  indicate  the  predominant  type  of  work  being  done  under  this 
contract. 

1  □  Advanced  development 

2  □  Eng  ineering  development 

3  i _ [Low  rate  production 

~  4  ' _ I  Production 

5  D  Other 
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Please  indicate  the  category  of  weapon/ equipment  system  which  best 
describes  this  contract: 

1  O  Aircraft 

2  O  Electronics 

3  [j  Missile 

Check  4  Q  Ordnance 

only 

one  5  □  Ship 

6  O  Space 

7  Q Surface  vehicle 

8  Q  Other  (Please  specify):  _ 


5.  what  is  the  approximate  total  dollar  value  of  this  contract? 

1  □  Under  SIGN 

2  D  At  least  S1CM  but  less  than  S25M 

3  Q  At  least  525M  but  less  than  $4CM 

CJ:eck  __ 

only  4  □  At  least  $4CM  but  less  than  S1CCM 
me 

5  Ej  At  least  S10CM  but  less  than  S160M 

6  O  At  least  S1SCM  but  less  than  S5C0M 

7  Qj  S500M  or  more 

6.  What  type  of  contract  is  the  one  you  have  selected? 

1  □CPFF 

2  □  CPAF 

7?  teak 

only  3  DCPIF 

one  ^ 

4  □rPI 

5  lU  Other  (Piease  soecifv):  _ _ 
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What  is  the  duration  of  the  contract? 

1  Q  Less  than  12  months 

2  n  At  least  12  months  but  less  than  24  months 

Check  3  Q  At  least  24  months  but  less  than  36  months 

only 

ggg.  4  Q  At  least  36  months  but  less  than  48  months 

5  D  At  least  48  months  but  less  than  60  months 

6  D  60  months  or  more 


8.  Was  the  contract  definitized  at  the  time  of  contract  award? 

Check  1  □  Ves - >»SKIP  TO  Q.10 

one  , — , 

-  /2  □  No 


IF  "NO" 

9.  Approximately  how  many  months  elapsed  from  contract  award  to 
contract  defini tization? 

1  O  Less  than  3  months 

2  CU  At  least  3  months  but  less  than  6 

Check  3  Q  At  least  6  months  but  less  than  9 

only 

£££  4  Q  At  least  9  months  but  less  than  12 

5  CD  At  least  12  months  but  less  than  15 

6  D  At  least  15  months  but  less  than  18 

7  Q  18  months  or  more 


f 


A 


10.  Approximately  how  many  months  after  the  contract  award  was  the  C/SCSC 
performance  measurement  baseline  established? 

1  Q  Less  than  3  months 

2  Q  At  least  3  months  but  less  than  6 

3  Q  At  least  6  months  but  less  than  9 

4  G  At  least  9  months  but  less  than  12 

5  O  At  least  12  months  but  less  than  15 

6  CD  At  least  15  months  but  less  than  18 

7  Q  At  least  18  months  but  less  than  21 

8  O  At  least  21  months  but  less  than  24 

9  Q  24  months  or  more 


L.-:ecK 

only 

one 


The  next  series  of  questions  deal  with  the  DoD  C/SCSC  review  process  that 
normally  entails  several  in-plant  visits  by  DoD  C/SCSC  review  teams. 


The  Question  11  series  should  be  answered  in  reference  only  to  the  one  visit 
that  you  consider  to  have  been  the  key  DoD  C/SCSC  review  for  this  contract. 


;Z3 sc.  on  -jcicn  rec: 


'lleciicn  o-'  i he  key  review  .  . 


11a.  Approximately  how  many  months  after  contract  award  was  the  Demonstra¬ 
tion  Review  or  the  Subsequent  Application  Review  held? 

1  G  Less  than  3  months 

2  0  At  least  3  months  but  less  than  6 

3  Cj  At  least  6  months  but  less  than  12 

4  O  At  least  12  months  but  less  than  18 

5  Q  At  least  18  months  but  less  than  24 

6  G  At  least  24  months  but  less  than  30 

7  G  At  least  30  months  but  less  than  25 

8  Q  36  months  or  more 


u  r.ec  < 

only 

one 
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lib.  What  organizations  were  represented? 
1  0  Program  Office 


Check 

all 

that 


2  0  Service  C/SCSC  Focal  Point,  i.e.,  HQ  DARCOM, 

HQ  NAVMAT,  HQ  AFSC 

3  0  Government  plant  representatives,  e.g.,  DCAS, 

AFPRO,  NAVPRO,  etc. 


avvlu  i — i 

4  | _ I  Commodity  Command,  e.g.,  MICOM,  ASD,  NAVAIR,  etc 

5  0  OCAA 

6  0  Other  (Please  specify):  _ 

11c.  What  was  the  duration  of  the  review  in  workdays? 

1  0  Under  3  workdays 

_  .  20  3-5  workdays 

Check 

:K'y  3  0  6-10  workdays 

CK£ 

4  0  11-15  workdays 

5  0  More  than  15  workdays 

lid.  What  was  the  size  of  the  Review  Team? 

1  0  Less  than  4  persons 
20  4-7  persons 
3  0  8-11  persons 

Check 

o”'*  4  0  12-15  persons 

one 

5  0  16-19  persons 

6  0  20-23  persons 

7  0  24  or  more  persons 


lie.  Approximately  how  many  discrepancies  or  corrective  action  items  were 
identified  by  the  DoD  Review  Team? 

1  Q  10  or  1  ess 

2  □  n-:o 

3  □  21-30 

Cneok  ~~ 

cnly  4  □  31-40 

one 

5  □  41-50 

6  C]  More  than  50 

7  □  Don't  know,  too  difficult  to  approximate 


Ilf.  How  would  you  rate  the  review  team's  performance  on  each  of  the 
following  characteristics  using  the  scale  provided? 


Cneak  one  hex  for 

each.  -Lien 

Excel  1 ent 

Good 

Fair 

Poor 

Ho 

Ooinion 

a. 

Team  leadership  and  control 

iD 

20 

3D 

4  □ 

5  □ 

b. 

Technical  qualifications  of 
the  team 

iD 

zD 

3  □ 

4  □ 

5  D 

c. 

Team  working  relationships 

iD 

2  □ 

30 

4  □ 

with  your  company 

5  LJ 

d. 

Thoroughness  of  team's  review 

in 

2  □ 

3Q 

4  □ 

50 

e. 

Team's  use  of  common  sense 
and  flexibility 

iD 

zD 

3  □ 

4  □ 

50 

f. 

Overall  team  professional  ism 
and  effectiveness 

iD 

2  □ 

3  □ 

4  □ 

a 

in 

llg.  To  the  best  of  your  knowledge,  were  this  review  team's  C/SCSC  interpre¬ 
tations  reasonably  consistent  with  other  DoD  review  teams'  interoreta- 
tions? 


Cr.ecx. 

zkZ'j 

one 


1  □  Yes 

2  □  Ho 

3  O  Ho  opinion 
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12.  Based  on  your  recollection  of  the  entire  review  process,  please  estimate 
how  many  man-days  were  spent  by  your  company  in  support  of  the  entire 
review  process  over  and  above  the  normal  expenditures  for  the  day-to-day 
operation  of  your  cost  and  schedule  control  system.  (Do  not  include 
work  devoted  to  system  implementation,  establishing  the  performance 
measurement  baseline  or  for  operating  the  internal  cost/schedule  control 
system. ) 

1  □  0  -  10  man-days 

2  l~~|  11  -  25  man-days 

3  □  26  -  50  man-days 

_  ,  4  0  51  -  100  man-days 

CkecK 

only  5  □  101  -  150  man-days 

one 

6  O  151  -  200  mart-day: 

7  Q  201  -  250  man-days 

8  □  251  -  300  man-days 

9  O  Over  300  man-days 


13.  In  your  opinion,  did  the  benefits  to  your  company  from  the  review  pro¬ 
cess  outweigh  its  costs  to  your  company? 


Check 

only 

one 


1  □  Yes 

2  □  No 

3  Q  No  opinion 
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14.  Approximately  how  many  months  after  contract  award  did  you  receive 
official  notification  from  your  contracting  office  that  your  system  had 
been  satisfactorily  applied  to  the  contract? 

1  O  Less  than  6  months 

2  O  At  least  6  months  but  under  12  months 

3  O  At  least  12  months  but  under  18  months 

4  D  At  least  13  months  but  under  24  months 

Check  5  D  At  least  24  months  but  under  30  months 
only 

one  6  D  At  least  30  months  but  under  36  months 

7  n  At  least  36  months  but  under  42  months 

8  □  At  least  42  months  but  under  43  months 

9  CJ  48  months  or  more 

15.  In  what  year  was  this  notification  received? 

1  □  1983 

2  □  1982 

3  □  1981 

Gvck  4  □  I960 

£?  5  □ 1979 

6  □  1978 

7  □  1977 

8  □  1976 

9  Q  1975  or  before 
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The  following  series  of  questions  deal  with  Cost  Performance  Reports  and 
your  opinions  concerning  CPRs. 


16.  Considering  the  contract  itself  as  CWBS  Level  1,  what  is  the  lowest  CWBS 
level  regularly  reported  by  you  to  the  government  on  Format  1  of  the 
Cost  Performance  Report? 


Check 

only 

one 


1  □  Level 

2  0  Level 

3  □  Level 

4  0  Level 

5  0  Level 

6  0  Level 

7  0  Level 

8  0  Level 


2 

3 

4 

5 

6 

7 

8 

9  or  lower 


17.  Do  you  consider  the  level  of  reporting  required  under  your  contract  in 
terms  of  its  depth  within  the  CWBS  to  be  .  .  .? 


Check 

only 

one 


1  0  TOO  low 

2  0  Too  high 

3  O  Just  about  right 

4  0  No  opinion 


18.  In  your  opinion,  how  effectively  does  the  DoD  Program  Office  to  which 
you  submit  Cost  Performance  Reports  use  the  data? 


Check 

only 

one 


1  0  Very  effectively 

2  0  Somewhat  effectively 

3  0  Not  at  all  effectively 

4  0  No  opinion 
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For  each  of  the  following  report  format  areas,  please  indicate  the 
extent  to  which  you  believe  the  data  requested  by  the  DoD  Program  Office 
is  excessive  to  the  DoD  Program  Office's  needs: 


19. 


Check  one  hoc:  for  each  izer. 


a. 

Format  1 

-  CWBS  1  eve! s 

b. 

Format  2 

-  Organization  levels 

c. 

Format  3 

-  Baseline  information 

d. 

Format  4 

-  Manpower  information 

e. 

Format  5 

-  Problem  analysis 

Extremely  Somewhat  Not 


Excessive 

Excessive 

Excessive 

1  □ 

zD 

?□ 

iU 

zQ 

3a 

1  □ 

2  □ 

3  □ 

iD 

zD 

3Q 

i  □ 

zD 

sQ 

20.  In  your  opinion,  could  the  number  of  pages  of  internal  reports  generated 
each  month  from  your  C/SCSC-accepted  system  be  reouced  without  harr.cering 
your  ability  to  manage  the  contract? 
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In  your  view,  how  accurately  (for  the  "as  of"  date  of  the  report)  dees 
the  Cost  Performance  Report  portray  the  cost  and  schedule  status  of  you 
contract? 

1  CH  Very  accurately 

Check  2  □  Somewhat  accurately 

only 

cr.e  3  I _ 1  Not  too  accurately 

4  CU  Not  at  all  accurately 


How  often  does  your  company  Program  Manager  personally  review  reports 
of  any  type  either  generated  directly  by  your  C/SCSC-accepted  system 
or  derived  from  your  system? 

1  [J  Daily 

2  Q  Weekly 

3  Q  Biweekly 

Cr.eck 

only  4  Q  Monthly 

one 

5  □  Quarterly 

6  Q  Less  often  than  quarterly 

7  Q  Never 


Approximately  how  many  calendar  weeks  after  the  close  of  your  monthly 
reporting  period  does  your  company  provide  C/SCSC-rel ated  reports  to 
your  Cost  Account  Managers? 

1  Q  Less  than  1  week 

2  Q  At  least  1  week  but  less  than  2 

Cneck 

only  3  L3  At  least  2  weeks  but  less  than  3 

one 

4  Q  At  least  3  weeks  but  less  than  4 

5  []  4  weeks  or  more 


*  o 


25.  Approximately  how  many  calendar  weeks  after  the  close  of  your  monthly 
reporting  period  does  your  company  forward  Cost  Performance  Reports  to 
the  government? 

1  f~|  Less  than  1  week 

2  0  At  least  1  week  but  less  than  2 

Check  3  0  At  least  2  weeks  but  less  than  3 

only 

one  4  0  At  least  3  weeks  but  less  than  4 

5  0  At  least  4  weeks  but  less  than  5 
6[]  5  weeks  or  more 

26.  Approximately  how  many  man-years  per  year  of  all  categories  of  employees 
are  spent  on  operating  your  C/SCSC-accepted  system  for  this  contract  and 
analyzing  its  ou^out? 

1  []  2  man-years  or  less 

2  0  3-  5  man-years 
30  a  -  10  man-years 

Check 

only  4  0  11  -  15  man-years 

one  ' 

~  50  16  -  20  man-years 

6  0  21  -  25  man-years 

7  0  25  -  30  man-years 

8  0  31  -  50  man-years 

9  0  Over  50  man-years 

10  0  Don't  know,  too  difficult  to  estimate 
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Please  indicate  whether  you  agree,  disagree,  or  have  no  opinion  concerning 
the  following  statements  in  questions  27  through  34: 

27.  The  in-plant  government  people  such  as  DCAS  or  AFPROs  have  a  variety  of 
responsibilities,  with  C/SCSC  usually  being  very  low  on  their  priority 
list. 


Check 

only 

one 


1 Q  Agree  strongly 
2D  Agree  somewhat 

3  0  Disagree  somewhat 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 


28.  The  in-plant  government  people  do  not  have  much,  if  any,  expertise  in 
C/SCSC. 

1  0  Agree  strongly 


Check 

only 

one 


2  0  Agree  somewhat 

3  0  Disagree  somewhat 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 


29.  C/SCSC-driven  internal  cost  and  schedule  reports  are  received  on  a  suf¬ 
ficiently  timely  basis  so  as  to  be  useful  to  your  Program  Office  in 
determining  cost  and/or  schedule  status. 

1  0  Agree  strongly 

2  0  Agree  somewhat 

Check 

only  3  0  Disagree  somewhat 
one 

4  0  Disagree  strongly 

5  0  Neither  agree  nor  disagree 
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3G.  DoD  C/SCSC  practices  for  this  contract  were  not  effective  in  ensuring 
that  an  adequate  basis  existed  in  your  company's  system  for  reporting 
cost  and  schedule  status. 


Check 

only 

c*ie 


1  Q  Agree  strongly 

2  Q  Agree  somewhat 

3  Q  Disagree  somewhat 

4  Q  Disagree  strongly 

5  [^Neither  agree  nor  disagree 


31.  The  costs  to  your  company  (and  to  DoD)  in  terms  of  time  and  manpower 
expended  to  operate  the  C/SCSC-accepted  system  are  outweighed  by  the 
benefits  derived  by  your  Program  Office  in  managing  the  contract. 


Check 

cnly 

cne 


1  Q  Agree  strongly 

2  Q  Agree  somewhat 

3  Q  Disagree  somewhat 

4  [^[Disagree  strongly 

5  |  |  Neither  agree  nor  disagree 


32.  The  concept  of  using  criteria  on  what  is  an  acceptable  cose  and  schedule 
control  system  rather  than  specifying  a  single  system  for  all  contrac¬ 
tors  to  use  is  not  valid. 


Check 

only 

cr.e 


iQ  Agree  strongly 

2  Q  Agree  somewhat 

3  Q  Disagree  somewhat 

4  Q  Disagree  strongly 

5  Q  Neither  agree  nor  disagree 
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The  C/SCSC  approach  of  ensuring  an  adequate  basis  for  keeping  your 
Program  Office  informed  on  the  cost  and  schedule  status  of  a  contract 
is  as  good  or  better  than  other  approaches  to  achieving  the  same 
objectives. 

1  Q  Agree  strongly 

2  Q  Agree  somewhat 

Check 

only  3  Q  Disagree  somewhat 
one 

4  [J  Disagree  strongly 

5  Q  Neither  agree  nor  disagree 

The  specific  cost  and  schedule  control  systems  criteria  as  currently 
delineated  by  DoD  are  not  appropriate  for  evaluating  whether  your 
company's  system  provides  an  adequate  basis  for  properly  indicating 
cost  and  schedule  status  on  this  type  of  contract. 

1  Q  Agree  strongly 

2  Q  Agree  somewhat 

3  Q  Disagree  somewnat 

— -  4  Q  Disagree  strongly 

5  Q  Neither  agree  nor  disagree 


To  the  best  of  your  knowledge,  how  frequently  does  the  government  C/SCSC 
Surveillance  Monitor  have  contact  with  your  company? 

1  □  Daily 

2  Q  Weekly 

Che.CK  3  □  Monthly 
only  l — l 

4  Q  Less  often  than  monthly 

5  Q  Don 1 1  know 


36.  Does  your  company  have  an  in-house  recurring  training  program  for 
managers  on  how  to  use  internal  reports  from  your  C/SCSC-accepted 
system? 

~nesk  1  □ 

—  2  Q  NO 
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Finally,  please  give  us  your  thoughts  on  what  steps  DoD  might  take  to 
improve  the  usefulness  of  C/SCSC  to  you  and/or  to  the  DoD  itself.  If 
you  wish  to  comment  about  any  other  aspect  of  C/SCSC,  we  welcome  suc.n 
observations.  These  comments,  as  well  as  all  other  information  provided 
in  this  questionnaire,  will  remain  completely  confidential .  Your 
responses  will  never  be  associated  with  any  information  by  which  you  can 
be  personally  identified.  If  you  choose  to  rescond  —  and  we  nooe  you 
will  --  please  use  the  space  below  and  feel  free  to  ado  extra  pages  as 
necessary.  We  would  appreciate  hearing  from  you. 
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APPENDIX  2 


COMPARATIVE  TABULATION  OF  RESPONSES 


This  appendix  contains  the  tabulation  of  responses  for  the  four  (4) 
questionnaires.  The  tabulations  are  listed  in  the  order  of  the  analyses 
contained  in  Section  III  and  the  summary  profiles  contained  in  Appendix  3. 

The  tabulations  for  the  analyses  contained  in  Section  III  start  on  page  A 2-2. 
The  tabulations  for  the  summary  contained  in  Appendix  3  start  on  page  A2-44. 


A2  -  1 


i 


C/SCSC  SURVEY 

TABULATION  &  COMPARISON  OF  RESPONSES 


* 


C/SCSC  SURVEY 

TABULATION  &  COMPARISON  OF  RESPONSES 
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Although  government  program  manager  question  -es  were  mailed  to  28  Army  program  managers,  29 
of  the  returned  government  program  manager  qi,.  jtionnaires  indicated  that  the  department  respons.jle 
for  the  respondent's  program  was  the  Army. 
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APPENDIX  3 


RESPONDENT  PROFILES 


The  four  (4)  questionnaires  contained  questions  whose  answers  provided 
profiles  of  the  respondents.  These  questions  included,  for  example, 
responsible  military  service,  phase  of  program,  category  of  weapon  system, 
etc.  Some  of  these  questions  also  were  useful  in  cross-tabulations  with 
responses  to  other  questions.  A  complete  tabulation  of  responses  is  included 
in  Appendix  2.  The  following  is  a  summary  which  may  provide  the  reader  with 
useful  profiles  of  the  four  (4)  respondent  populations. 

Respondent  tabulations  show  the  following: 

More  of  the  programs/contracts  were  in  research  and  development 
than  in  production. 

Approximately  70%  to  75%  of  the  programs/contracts  were  in  the 
aircraft,  electronics  or  missile  weapons  systems  categories. 

Two-thirds  167%)  of  the  government  program  managers  estimated  the 
approximate  cost  of  their  programs  as  $1  billion  or  more. 
Approximately  85%  of  the  contracts  with  the  C/SCSC  requirement  had 
a  total  dollar  value  of  at  least  $40  million. 

Approximately  the  same  percentage  of  contracts  with  C/SCSC  were 
cost-plus  and  fixed-price  incentive. 

Approximately  one-third  of  the  contracts  with  C/SCSC  were  not 
definitized  at  contract  award. 

Of  those  contracts  not  definitized  at  contract  award,  approximately 
60%  to  70%  were  reported  to  have  been  definitized  within  12  months. 

Government  and  contractor  program  managers  were  asked  to  indicate 
the  extent  of  their  personal  knowledge  of  five  (5)  CPR  formats. 
Overall,  contractor  managers  rated  themselves  as  more  knowledgeable 
than  government  managers  rated  themselves. 

-  Contractors  for  approximately  two-thirds  of  the  contracts 

responding  to  the  survey  received  their  notification  in  1981,  1982 
or  1983  that  they  had  met  the  C/SCSC  requirement  for  their  contract. 

Most  government  program  managers  (71%)  have  been  in  their  current 
jobs  for  3  years  or  less;  while  the  majority  of  contractor  program 
managers  (64%)  have  been  in  their  current  jobs  for  more  than  3 
years. 
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Most  contractor  program  managers  (75%)  and  most  government  program 
managers  (59%)  had  previous  jobs  which  provided  them  some 
familiarity  with  C/SCSC. 

The  majority  of  government  program  managers  (52%)  had  received 
C/SCSC  training  at  DGD  training  courses  (Defense  Systems  Management 
College,  Air  Force  Institute  of  Technology  or  Army  Management 
Engineering  Training  Agency).  Most  contractor  program  managers 
(89%)  received  their  C/SCSC  training  from  in-house  and  on-the-job 
opportunities. 
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